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concentrated  mainly  on  estimated  savings  from  competition  at 
the  time  of  contract  award.  This  study  expanded  prior 
analysis  by  Investigating  the  cost  and  schedule  growth  over 
the  contract's  period  of  performance. 

When  comparing  competed  and  sole  source  contracts  the 
results  ware  consistent/  with  sole  source  contracts 
exhibiting  higher  coat  growths  in  all  areas.  The  sole 
source  contracts  also  exhibited  a  higher  schedule  growth. 
Between  lower  and  higher  initial  priced  contracts  the 
results  were  also  consistent  with  higher  priced  contracts 
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The  purpose  of  this  study  was  to  determine  if 
differences  exist  in  the  cost  and  schedule  growth  of 
competed  versus  sole  source  contracts.  Prior  studies 
concentrated  mainly  on  estimated  savings  from  competition  at 
the  time  of  contract  award.  This  study  expanded  prior 
analysis  by  investigating  the  cost  and  schedule  growth  over 
the  contract's  period  of  performance. 

In  addition  to  an  analysis  of  cost  and  schedule  growth 
between  competed  and  sole  source  contracts^  the  cost  growth 
analysis  also  investigated  differences  between  lower  and 
higher  risk  contracts  as  well  as  lower  and  higher  initial 
priced  contracts.  The  results  were  consistent  in  every 
category  with  sole  source  contracts  showing  a  greater  cost 
and  schedule  growth  than  competed  contracts. 

When  comparing  competed  and  sole  source  contracts,  the 
results  were  consistent  with  sole  source  contracts 
exhibiting  an  average  of  57%  higher  cost  growth  in  all 
areas.  Sole  source  contracts  also  exhibited  a  higher 
schedule  growth  that  was  over  4  times  greater  than  the 
schedule  grcwth  of  competed  contracts. 

When  comparing  lower  and  higher  initial  priced 
contracts,  the  results  were  also  consistent  with  higher 
priced  contracts  exhibiting  higher  cost  growth  than  lower 
price  contracts .  The  results  were  mixed  when  measured 
between  lower  and  higher  risk  contracts  however. 
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ANALYSIS  OF  COST  AND  SCHEDULE  GROWTH  ON 
SOLE  SOURCE  AND  COMPETITIVE 
AIR  FORCE  CONTRACTS 

Introduotlon 

Chapter  Overview 

This  chapter  describes  the  general  issue  and  specific 
problem  researched  by  this  thesis.  Also,  the  research 
objectives,  investigative  questions,  hypotheses  and  research 
variables  are  explained.  Finally,  the  scope  and  limitations 
of  the  research  are  defined. 

General  Issue 

Competition  in  defense  acquisitions  has  long  been 

recognized  as  a  means  to  improving  performance,  reducing 

risk,  benefiting  schedule,  improving  quality,  strengthening 

the  defense  industrial  base,  and  reducing  acquisition  costs 

(9:1-1).  Although  these  benefits  seem  reasonable,  all  have 

been  somewhat  difficult  to  measure.  Because  cost  savings 

are  easier  to  measure  than  the  other  parameters,  it  has 

become  the  focus  of  research  on  the  effects  of  competition. 

The  emphasis  on  cost  savings  as  the  primary  measure  of  the 

benefits  of  competition  began  in  the  early  1960's  during  the 

tenure  of  Secretary  of  Defense  McNamara, 

In  1965,  the  then  Secretary  of  Defense  Robert  S. 
McNamara  indicated  to  the  Joint  Economic  Committee 
(Hearings  to  the  Economic  Impact  of  Federal 
Procurement)  that  the  General  Accounting  Office  (GAO) 
had  evidence  of  dollar  savings  on  the  order  of  25 
percent  or  more  when  competition  was  introduced  for  re¬ 
procurement  of  an  item  which  had  a  sole-source 
procurement  history.  (3:2) 
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This  figura  of  25  percent  has  become  a  loosely  used 
benchmark  of  tha  benefits  of  competition.  This  position  has 
consistently  been  reaffirmed  by  past  and  present 
administrations  and  Congress.  More  recently/  Public  Law  98- 
369  implemented  the  Competition  In  Contracting  Act  (1984) . 
This  act  mandates  tha  use  of  competition  to  the  fullest 
extent  possible  on  all  government  procurements.  Since 
McNamara's  claim,  research  efforts  have  attempted  to  measure 
the  cost  savings  achieved  through  competition.  Measuring 
the  savings  of  competitive  versus  sole  source  acquisitions 
is  no  trivial  task.  Numerous  costs  to  develop  and  maintain 
competition  must  be  accurately  measured  to  calculate  any 
savings  due  to  competition  over  sole  source  awards. 

Beltramo  Identified  several  categories  of  costs  necessary  to 
accurately  measure  savings  due  to  competition  (1:16).  These 
categories  are: 

Technology  transfer  to  second  source 
Building  educational  units 
Additional  government  management 
Legal  claims  against  the  government 
Time  value  of  money 

This  Is  not  an  exhaustive  list  of  the  added  costs,  but 
serves  to  illustrate  the  complexity  Involved  In  accurately 
measuring  the  "costs  of  competition". 

Separate  research  by  Beltramo  (1:16);  Boger  (4:49),  and 
Berg  (2:12)  showed  general  agreement  on  tha  categories  of 
costs  Incurred  by  the  government  to  compete  a  procurement, 
but  disagree  on  the  methodologies  used  to  measure  the  direct 
contract  award  savings  attributed  to  the  Introduction  of 
competition,  and  any  savings  resultlitg  from  competition. 

The  methodologies  developed  In  prior  studies  Investigating 
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the  "coat  of  competition"  and  savings  from  competition  have 
exclusively  used  vreapon  systems  that  had  a  period  of  sole 
source  production  followed  by  competition.  These  weapon 
system  contracts  were  used  to  calculate  cost  differences 
between  sole  source  and  competitive  procurements. 

The  controversies  on  methodology  primarily  focus  on  how 
each  research  study  calculates  the  sole  source  and 
competitive  unit  cost.  Due  to  researchers'  inability  to 
develop  a  standard  methodology,  the  claimed  cost  savings 
attributed  to  competition  in  most  studies  have  large 
variations  (10:16).  Chapter  II  discusses  the  differences  in 
research  methodologies  in  prior  studies  that  create 
variations  measuring  cost  savings.  If  the  methodology  used 
to  calculate  cost  savings  can  be  agreed  upon  by  the 
researchers  studying  competition,  the  DoD  may  be  able  to 
predict  the  cost  reduction  achieved  with  competition  at  the 
time  of  contract  award.  However,  cost  savings  at  the  time 
of  contract  award  is  not  the  only  relevant  measurement  to 
determine  the  benefits  of  competition.  For  the  cost  aspects 
of  a  program  a  more  pertinent  measurement  may  be  the  total 
program  cost.  Also,  since  performance  schedules  are 
critical  source  selection  criteria,  the  initial  schedules 
bid  by  offerors  cannot  be  evaluated  as  a  benefit  of 
competition  unless  the  actual  delivery  performances  on  sole- 
source  and  competitive  acquisitions  are  evaluated. 

Specific  Problem 

Research  to  date  has  not  examined  the  cost  and  schedule 
growth  during  the  execution  of  the  contract  for  sole  source 
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and  competitive  awards .  Research  on  the  benefits  of 
oompetltion  has  either  ignored  any  differences  in  cost  and 
schedule  growth  between  competed  and  sole  sourced  contracts 
or  has  assumed  no  difference.  If  cost  and  schedule  growth 
differences  do  exist  between  competed  and  sole  source 
contracts,  then  these  differences  must  be  considered  before 
claims  can  be  made  on  the  benefits  of  competition. 

Qb.1.t5itiyg9 

The  objective  of  this  research  is  to  determine  if  cost 
and  schedule  growth  rates  are  different  on  competitive  and 
sola  source  contract  awards.  Also,  the  research  will 
determine  if  the  differences  in  growth  rates  oh  sole  source 
and  competitive  acquisitions  are  statistically  significant 
for  different  program  risk  levels  and  initial  contract  award 
price  levels.  Contract  risk  is  identified  in  this  study  by 
the  type  of  contract  used  by  the  government.  It  is 
generally  assumed  that  cost  type  contracts  are  higher  risk 
than  fixed  priced  contracts.  Therefore,  the  higher  risk 
category  is  comprised  of  coat-~type  contracts,  while  the 
lower  risk  category  is  comprised  of  fixed  price  contracts. 
The  initial  contract  price  is  simply  the  total  award  price 
of  the  contract.  The  sample  contracts  are  stratified  by 
price  level  in  order  to  determine  if  high  priced  efforts 
have  significantly  different  cost  growth  than  lower  priced 
awards . 

By  comparing  the  behavior  of  cost  and  schedule  growth 
between  sola  source  and  competed  contracts,  this  study  will 
add  to  the  existing  body  of  knowledge  on  the  effects  of 
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competition.  Measuring  the  cost  growth  rates  relative  to 
contract  risk  gives  the  Air  Force  deeper  insight  into  the 
effects  of  risk  on  contract  cost  growth.  By  providing  the 
Air  Force  with  an  analysis  on  competition  that  considers 
program  risk^  this  research  can  help  decision  makers  develop 
acquisition  strategies  and  help  select  the  appropriate  type 
of  contract.  Measuring  cost  growth  rates  relative  to 
initial  contract  price  determines  if  the  dollar  amount  of  a 
contract  is  related  to  the  level  of  competition.  At  a 
minimum/  the  analysis  can  assist  the  Air  Force  developed 
budgets  and  schedules  based  on  initial  dollar  amounts  and 
program  risk  level . 

RM«argh,Quflatlona  .and  Hypothtaia 

To  answer  the  research  objective/  the  following  three 
research  questions  were  Investigated  and  their  associated 
hypotheses  were  tested. 

Research  Question  #1.  For  cost  growth  and  schedule 
growth  categories/  does  the  mean  growth  for  a  competed 
contract  differ  from  the  mean  growth  of  a  sole  source 
contract? 

HyB<?VfhagAa., 

Hoi  Moan  Competed  Growth  •  Mean  Sole  source  Growth 
Hai  Mean  Competed  Growth  *  Mean  Sole  Source  Growth 


Research  Question  #2.  For  the  cost  growth  category 
only/  does  the  mean  cost  growth  for  competed  and  sole  source 
lower  risk  contracts  differ  from  competed  and  sole  source 
higher  risk  contracts? 


5 


Hypothesis. 


Hoi  Low  Riak  naan  Growth  -  Hiffh  Risk  Maan  Growth 
Ha  I  Low  Riak  Maan  Growth  *  High  Risk  Maan  Growth 


Rutigch  Vox  the  oost  growth  category 

only,  doei  the  mean  coat  growth  for  oompeted  and  sole  source 
lower  prloe  contracts  differ  from  oompeted  and  sole  source 
higher  prloe  contracts? 

Hoi  LOW  priea  Maan  Growth  «  High  Price  Afean  Growth 
Hai  Low  Priea  Maan  Growth  #  High  Price  Maan  Growth 


These  research  questions  stem  from  the  acquisition 
phenomenon  of  "buying-in"  to  the  contract.  Buying  in  is 
simply  the  contractor^  a  tendency  to  bid  overly  optimistic 
costs  and  schedules  In  hopes  of  winning  an  acquisition 
competition. 

In  the  oompetitive  bidding  for  the  development 
contract,  oontractors  frequently  submit  bids  below 
their  expected  cost,  with  the  knowledge  that  price 
will  bo  renegotiated  over  time  to  accommodate 
engineering  change  proposals  that  modify  the 
original  design.  (10:58) 

Contractors  on  sole  source  acquisitions  may  not  feel 
the  pressure  to  bid  aggressive  schedules  and  optimistic 
costs  on  new  work.  Based  on  the  buying-in  premise,  oost  and 
schedule  growth  may  bo  different  on  sole  source  and 
competitive  acquisitions.  Figure  1  depicts  the  anticipated 
difference  in  cost  growths  for  sole  source  and  competitive 
awards. 
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In  Figura  1,  th«  points  "A"  and  "B"  rsprssent  ths 
hypothstiosl  Initial  oontraat  prioas  for  a  sola  aouroa  and 
oompatltlva  award.  Tha  solid  Unas  to  tha  right  from  points 
"A"  and  "B"  Indloatad  an  antloipatad  growth  In  contract 
prlca  during  tha  parlod  of  parformanoa  of  tha  contract.  Xt 
suggests  that  tha  compatad  contract  price  grows  at  a  faster 
rata  than  tha  sola  source  award.  Although  tha  Initial 
competition  seamed  to  yield  a  significant  cost  savings/  in 
ths  final  analysis/  no  savings  wars  aohlsvad.  This  Is 
purely  for  demonstration  and  not  a  claim. 

Lika  contract  costs /  ths  schedules  proposed  on 
competitive  acquisitions/  may  be  optimistic.  "Competition 
advocates  claim  that  competition  Improves  the  schedule 
deliveries  for  defense  acquisitions."  (4:69).  Like  costs, 
researchers  merely  compare  the.  schedules  proposed  on 
competitive  and  sole  source  contracts  at  the  time  of 
contract  award.  If  tha  schedule  growth  on  competed 
contracts  grows  at  a  higher  rate  than  sole  source  awards. 
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Figure  1  may  also  depict  how  the  initial  schedule  benefits 
claimed  at  the  time  of  contract  award  dissipate  over  time. 

Research  Variables 

Cost  Growth.  To  complete  the  research  objectives  and 
test  the  cost  growth  hypotheses,  a  sample  mean  cost  growth 
factor  was  calculated  for  each  category  in  3  phases  of 
tests.  The  first  phase  tested  the  hypothesis  for  the 
overall  categories  of  competed  and  sole  source  contracts  as 
depicted  in  Table  3.  The  second  phase  tested  the  hypothesis 
after  the  competed  and  sole  source  data  were  segregated  Into 
lower  and  higher  rlsh  categories,  as  depicted  in  Table  4. 

The  third  phase  tested  the  hypothesis  after  the  lower  and 
higher  risk  data  were  segregated  into  lower  and  higher 
initial  price  as  depicted  in  Table  5. 

Schedule  Growth.  To  test  the  schedule  growth 
hypothesis,  a  sample  mean  schedule  growth  factor  was 
computed  from  a  random  sample  of  sole  source  and  competed 
contracts.  A  stratification  of  the  data  relative  to  risk 
and  initial  contract  price  similar  to  cost  growth  was  not 
possible  due  to  schedule  data  limitations. 

Scope 

The  research  focuses  on  cost  and  schedule  performar'^e 
on  Air  Force  Material  Command  development  and  production 
contracts  distributed  between  1980  and  1990  Inclusively. 

The  purpose  of  choosing  these  dates  Is  an  attempt  to 
mitigate  the  effects  of  the  defense  market  business  cycle. 
Boger  and  Greer  believed  that  the  price  of  defense 
development  and  production  was  somewhat  affected  by  the  idle 
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capacity  In  the  defense  industrial  base  (3:31).  Also,  this 
timeframe  distributes  the  sample  population  5  years  pre  CICA 
and  6  years  post  CICA.  By  evaluating  contracts  over  a  ten 
year  period,  this  research  attempts  to  eliminate  the  bias  on 
the  results  from  both  of  these  factors. 

To  investigate  the  research  questions  associated  with 
cost  growth,  data  was  gathered  from  contracts  managed  at: 

Los  Angeles  AFB,  CA;  Wright  Patterson  AFB,  OH;  Bglin  AFB,  FL 
and  Hanscom  AFB,  MA  as  well  as  Wright  Aeronautical 
Laboratories,  Wright  Patterson  AFB,  OH.  To  investigate  the 
research  question  for  schedule  growth,  contracts 
administered  by  the  Air  Force  Contract  Management  Canter  at 
Klrtland  AFB,  NM  were  used  as  the  sampling  population.  The 
purpose  of  ohoosing  the  populations  for  cost  and  schedule 
growth  analysis  was  that  they  represented  a  cross-section  of 
the  types  of  weapon  systems  the  Air  Force  procures. 

The  contract  information  that  contain  the  cost  and 
schedule  data  used  in  this  research  was  that  information 
captured  by  the  Acquisition  Management  Information  System 
(AMIS) .  The  AMIS  is  an  acquisition  management  operating 
system  concerned  with  pre-award  contract,  and  contract- 
related  data.  The  system  contains  information  on  all  AFMC 
research  and  development  contracts.  The  AMIS  stores  data  in 
twenty-six  on-line  data  bases.  The  two  data  bases  used  for 
this  research  were  the  CONTRACT  and  Program  Management 
System  (FMS)  data  bases.  The  CONTRACT  data  base  was  used  to 
gather  contract  schedule  information  while  the  PMS  data  base 
was  used  to  gather  cost  information. 
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The  CONTRACT  and  PMS  data  bases  allowed  the  researchers 
to  measure  mean  cost  and  schedule  growth  on  numerous 
contraots  without  access  to  the  actual  contract  files, 
Because  a  data  base  was  used  rather  than  original  contract 
documents,  the  data  is  subject  to  input  errors.  Also,  cost 
and  schedule  changes  due  to  contraot  scope  changes  can  not 
be  detected  without  referenoing  the  original  contract 
documents .  Since  this  researoh  is  simply  conducting  a 
comparative  study  of  cost  and  schedule  growth  on  sole  source 
and  competed  contraots,  it  la  assumed  that  these  affects 
will  be  the  same  on  each  category  under  Investigation. 

The  contracts  used  to  test  the  Investigative  questions 
on  schedule  growth  were  limited  to  efforts  administered  by 
the  Air  Force  Contract  Management  Center  at  Klrtland  AFB, 

MM,  This  population  was  used  because  schedule  Information 
on  these  contracts  was  more  readily  accessible  than  the 
contraote  controlled  by  other  contracting  centers.  This 
study  limited  the  contracts  considered  by  eliminating 
procurements  identified  as  service-type,  support,  and  basic 
researoh.  These  contracts  were  eliminated  from  the  sampling 
population  because  they  were  not  representative  of  Air  Force 
weapon  system  development  and  production  contracts. 

Also,  contracts  having  a  mixture  of  cost-type  and  fixed 
price  line  items  were  eliminated  because  they  were  not 
readily  identifiable  as  lower  or  higher  risk. 

.ggaslufljgn 

This  thesis  does  not  attempt  to  quantify  the  dollar 
savings  resulting  from  competition  nor  answer  questions 
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concerning  the  many  possible  benefits  of  competition.  This 
thesis  does  attempt  to  assess  if  differences  exist  in  the 
cost  and  schedule  growth  between  competed  and  sole  source 
contracts.  Chapter  II  provides  a  comprehensive  review  of 
previous  research  on  the  effectiveness  of  competition  and 
controversies  surrounding  the  competition  issue.  Chapter 
111  describes  the  methodology  used  to  meet  the  objectives  of 
this  thesis  and  answer  the  research  questions.  Chapter  IV 
presents  the  findings  and  analysis  of  the  cost  and  schedule 
growth  data  gathered  on  competitive  and  sole  source 
contracts.  Finally^  Chapter  V  provides  conclusions  based  on 
the  data  analysis  as  well  as  recommendations  for  future 
research. 
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Thtt  objaotlvtt  of  this  rasearch  is  to  detarmine  if  a 
diffaranoa  axista  in  oost  and  schadula  growth  batwaan  sola 
souroa  and  oompatad  Air  Foroa  acquisitions.  This  chapter 
first  highlights  the  background  of  tha  advocacy  for  the 
incraasad  usa  of  oompatition.  Tha  differing  viewpoints 
defining  oompatition  are  than  discussed  along  with  soma 
oritioisms  of  competition.  The  alternative  to  oompatition, 
sola  souroa,  is  than  presented.  Lastly,  tha  dual  source 
acquisition  strategy  is  examined  along  with  soma  of  its' 
oritioisms. 

This  cursory  overview  of  varying  acquisitions 
strategies,  along  with  tha  historical  development  of 
oompatition,  is  nacaasary  to  fully  understand  the 
signifioanoe  of  previous  research  studies.  Those  studies 
are  tha  basis  for  tha  savings  estimates  attributable  to 
competition.  Lastly,  the  savings  estimates  and  tha 
methodology  used  to  develop  them  are  discussed  in  detail. 

The  increased  push  for  competition  in  the  early  1960's, 
which  intensified  throughout  tha  1980' s,  was  supported  by 
various  studies  that  showed  competition  in  acquisitions 
produced  dollar  savings  on  tha  order  of  25%  (3:2).  The 
culmination  of  these  competition  initiatives  resulted  in  the 
passage  of  the  Competition  in  Contracting  Act  (CICA)  in 
1984,  which  mandated  the  use  of  effective  competition  for 


12 


the  procurement  of  defense  goods  and  services.  This  act 
specified  that  oontraots  should  be  awarded  on  the  basis  of 
best  value,  stating  that  in  many  oases,  quality  may  be  the 
dominant  factor  and  coat  may  be  secondary  in  source 
selection  decisions.  Assistant  Secretary  of  the  Air  ^oroe 
for  Acquisition,  John  T.  Welch  Jr.,  explained  in  1988  that 
the  purpose  behind  CICA  "was  not  to  establish  competition 
for  competition' a  sake,  but  to  foster  it  as  part  of  an 
overall  effort  to  achieve  the  best  value  for  our  taxpayer's 
dollars”  (5:67). 

The  current  mandates  for  Increased  use  of  competition 

in  weapon  system  acquisitions  exist  because  competition  is 

believed  to  provide  benefits  in  many  areas.  The  Department 

of  Defense  Directive  (OoDD)  5000.1  requires: 

Defense  systems,  subsystems,  equipment,  supplies 
and  services  shall  be  acquired  on  a  competitive 
basis  to  the  maximum  extent  practical  as  a  means 
of  achievinq  cost,  schedule,  and  performance 
benefits.  (g:1-6) 

Additional  benefits  from  competition  can  also  include: 
improving  quality,  strengthening  the  defense  industrial 
base,  enhancing  mobilization  or  surge  capability,  and 
reducing  risk.  Even  though  these  potential  benefits  were 
recognized,  in  1982,  only  34.3%  of  the  total  dollars  awarded 
were  awarded  competitively  (3.0:4). 

However,  since  the  implementation  of  CICA,  the  total 
dollar  value  and  relative  percentage  of  competed  dollars  has 
risen  steadily.  In  fiscal  year  1968,  60.9%  of  all 
contracting  dollars  were  awarded  as  a  result  of  competition, 
while  the  dollar  value  of  contracts  awarded  competitively 
nearly  doubled  from  fiscal  year  1984  to  1988.  Competitively 
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awarded  contracts  totaled  $12.4  billion  in  1984.  In  1988, 
that  number  rose  to  $23.6  billion  (5:67).  At  that  time, 

John  T.  Welch  Jr.  claimed  "unparalleled  success"  for 
implementing  CICA,  which  resulted  in  5  years  of  increased 
dollar  value  of  contracts  awarded  on  the  basis  of 
competitive  bidding"  (5:67). 

However,  it  may  be  misleading  to  measure  the  success  of 
CICA  based  upon  increased  dollar  values  of  contracts  awarded 
competitively.  The  1980' a  was  an  era  characterized  by 
massive  increases  in  defense  spending,  and  the  increased 
dollar  value  of  competed  contracts  may  simply  be 
attributable  to  the  increases  in  defense  spending. 

Wot  withstanding  the  dollar  level  and  percentage 
increases  in  competitively  awarded  contracts,  the  increased 
pressure  for  competition  demanded  an  accurate  definition  of 
what  constituted  competition.  This  definition  soon  became  a 
highly  criticized  issue. 

Competition  Issues 

Strong  criticism  of  the  services'  reporting  of 
competition  statistics  was  made  public  in  March  of  1989  with 
the  release  of  a  DoD  Inspector  General's  (IG)  report.  The 
hey  issue  raised  in  the  IG  report  was  the  question  of  what 
is  competition?  Anthony  DeLuca,  the  Air  Force  Competition 
Advocate  in  1969,  stated  "if  you  have  marketplace  forces  at 
work,  you  have  competition"  (11:42).  Taking  a  more 
pragmatic  view,  Joseph  F.  Grosson  and  Dr.  Joseph  H.  Augusta 
contend,  "to  have  competition,  there  must  be  a  second 
source,  i.e.,  at  least  one  alternate  producer  of  a  weapon 
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system"  (9:33).  Clearly,  this  viewpoint  takes  DeLuoa' s 
definition  one  step  further  by  stating  that  competition  only 
exists  when  more  than  one  actual  producer  of  a  weapon  system 
exists.  The  definition  of  what  constitutes  competition  is 
contested,  and  some  criticisms  surrounding  this  Issue  are 
discussed  below. 

in  the  i989  ig 

report,  the  IG  claimed  that  the  services  have  "grossly 
exaggerated"  competition  statistics  by  inflating  competition 
rates  by  as  much  as  50%  (11:42).  If  the  iG's  accusation  is 
true,  then  the  services  have  not  increased  the  level  of 
competition,  as  indicated  by  the  1988  figure  cited  earlier. 
At  the  center  of  the  controversy  was  the  determination  of 
what  is  competition.  The  IG  argued  that  when  marketplace 
forces  are  artificially  created  to  fulfill  the  need  for 
competition,  the  circumstances  surrounding  the  contract 
award  may  be  suspect  (11:42).  The  IG  auditors  took 
exception  to  the  services  classifying  some  contract  awards 
as  competitive.  Examples  cited  by  the  IG  were:  dual  source 
contracts  where  the  loser  would  still  expect  a  significant 
portion  of  the  work,  proposals  where  only  one  bid  was 
received,  and  the  classification  as  competitive,  those  sole 
source  follow-on  actions  where  the  original  contract  was 
competed.  One  highly  disputed  area  is  the  situation  where  a 
Request  for  Proposal  is  released  and  only  one  bidder 
responds.  Policy  guidance  from  the  Office  of  Federal 
Procurement  Policy  (OFPP)  directs  DoD  and  the  services  to 
report  such  contracts  as  competed  (11:44).  For  purposes  of 
this  research,  competition,  or  competitively  awarded 
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contracts,  are  considered  to  exist  only  where  two  or  more 
bids  were  received  in  response  to  a  solicitation.  The  views 
of  Grosson  and  Augusta,  that  there  must  be  a  second  source, 
are  the  views  adopted  here.  This  definition  will  become 
apparent  in  Chapter  III  where  the  methodology  surrounding 
the  classification  of  contracts  as  competitive  or  sole 
source  is  discussed. 

The  definition  of  what  constitutes  competition  is  only 
one  of  its'  criticisms.  Other  significant  criticisms 
discussed  in  previous  research  are  mentioned  here  for 
discussion  purposes. 

Other  Competition  Criticisms.  John  T.  Welch  Jr. 
explained  the  goal  of  CICA  was,  through  the  increased  use  of 
competition,  to  ensure  the  "best  value"  in  weapon  system 
procurements  for  the  taxpayer's  dollars  is  achieved  (5:67), 
Wot  withstanding  CICA's  reported  success,  criticisms  exist 
concerning  competition.  The  first  criticism  involves  cost 
and  pricing  data  while  the  second  criticism  is  how 
competition  may  be  limited  because  of  non-'competitive 
contract  awards  early  in  a  program's  development. 

Cost  and  Pricing  Data.  Under  federal  acquisition 
regulations,  cost  and  pricing  data  are  not  required  from  a 
contractor  if  competition  exists.  However,  by  erring  on  the 
side  of  labeling  contracts  as  competitive,  the  DoD  is 
compromising  one  of  its  key  safeguards  against  defective 
pricing  (11:42).  A  1988  Defense  Contract  Audit  Agency 
(DCAA)  study  reported  defective  pricing  in  almost  half  of 
the  non-competitive  contracts  audited.  Given  the  DCAA 
findings,  if  non-competitive  awards  are  incorrectly  being 
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labeled  as  competitive,  then  there  is  a  good  chance  that 

defective  pricing  is  occurring  at  an  alarming  rate.  Anthony 

DeLuca  aummed  it  up  well, 

If  market  forces  are  really  at  work,  then  you 
don't  need  cost  and  pricing  data.  But  if  you've 
created  a  mirage  of  competition,  and  you're  hiding 
behind  that  mirage  to  avoid  getting  cost  and 
^ricin^  data,  then  that's  clearly  a  mistake. 

With  the  debate  over  the  definition  of  what  constitutes 
competition  and  the  added  emphasis  for  increases  in 
competition,  contracts  are  being  labeled  as  competitive  when 
a  competitive  environment  did  not  exist.  As  the  previous 
audit  findings  indicate,  this  practice  limits  the 
government's  insight  into  a  contractor's  cost  and  pricing 
data  and  could  be  allowing  significant  pricing  errors  to 
continue  unnoticed. 

m  fiscal  year  1990, 

more  than  90%  of  the  DoD's  $6  billion  applied  research 

dollars  were  awarded  non-competitively  (14:45).  Applied 

research  typically  occurs  on  a  program  during  the  Concept 

Exploration  and  Demonstration  Validation  phases  and  defines 

the  technology  that  will  be  developed  in  later  phases  of  the 

program.  This  non-competitive  practice  establishes  DoD's 

design  preferences  early  in  the  program,  before  the  program 

is  competitively  solicited.  According  to  David  Soergel, 

Executive  Director  of  the  VEBEL  Society, 

This  leveling  of  pre-competitive  technology  and 
design  preferences  washes  out  substantive 
differences  between  proposals,  putting  an  undue 
emphasis  on  highly  uncertain  cost  estimates  as  the 
key  factor  in  award,  which  in  turn  motivates 
industrial  "price  buy-ins"  and  the 
politicalization  of  the  contract  award  process. 

(14:45) 
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with  the  reality  of  the  amount  of  non-competitive 
awards  for  a  majority  of  basic  research,  the  goal  of 
achieving  the  "best  value"  for  taxpayer's  dollars  may  be 
lost  when  the  proposed  cost  becomes  the  only  basis  for 
differentiating  between  competing  proposals.  In  other 
words,  the  technical  leveling  between  competing  alternatives 
further  increases  the  importance  and  emphasis  on  price. 

Even  with  the  emphasis  on  competitive  contracting,  awarding 
a  sole  source  contract  may  still  be  a  viable  alternative. 

Sole  Source  Contracts 

The  Competition  in  Contracting  Act  was  passed  to 
emphasize  competition,  but  allows  for  sole  source  contract 
awards  in  limited  ciroumstances.  Justification  for  a  sole 
source  contract  will  be  granted  if  any  one  of  the  following 
seven  exceptions  are  met: 

1.  There  is  only  one  source  available 

2.  There  is  a  unusual  and  oompelllng  urgency 

3.  Required  to  maintain  the  defense  industrial  base 

4.  It  is  required  by  treaty 

5.  It  is  required  by  statute 

6.  It  is  in  the  interests  of  national  security 

7.  It  is  in  the  public  Interest 

The  savings  estimates  of  25%  mentioned  earlier  were 
based  on  contracts  that  were  previously  sole  source,  and 
then  were  subsequently  competed  with  a  dual  source  strategy . 
The  next  section  discusses  dual  sourcing  and  some  of  that 
strategy's  criticisms.  Again,  an  understanding  of  this 
strategy  and  its'  criticisms  is  fundamental  to  an 
appreciation  of  the  savings  estimates  attributable  to 
competition,  that  were  derived  from  analysis  of  dual  source 
programs . 
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Dual  sourcing  ocoura  vrhen  two  or  more  producers  of  a 
particular  weapon  system  or  component  exist.  Recalling  that 
over  90%  of  DoD' a  applied  research  dollars  are  awarded  non- 
competitively,  then  the  only  recourse  to  Realizing 
competitive  savings  la  to  develop  a  second  (dual)  source  for 
development  or  production.  The  second  source  may  have  been 
developed  throughout  the  acquisition,  or  developed  in  later 
phases  of  the  program.  Developing  a  second  source  later  in 
the  program  can  be  used  to  stimulate  price  competition 
during  production  In  hopes  of  delivering  systems  at  a  lower 
price.  On  the  other  hand,  developing  dual  sources  early  In 
a  program  would  cause  head-to-head  competition  throughout 
development,  hopefully  producing  better  performing,  lower 
cost  weapon  systems,  at  reduced  development  risk,  while 
delivering  systems  sooner. 

Like  the  competitive  environment,  dual  sourcing  has 
several  criticisms.  Some  criticisms  are  now  discussed. 

Dual  Sourcing  Criticisms 

The  following  dual  sourcing  criticisms  are  not  meant  to 
be  an  exhaustive  list  of  all  possible  criticisms.  However, 
the  criticisms  mentioned  are  considered  to  significantly 
affect  the  cost  effectiveness  of  a  dual  source  strategy. 

The  cost  effectiveness  or  cost  savings  realized  from  a  dual 
source  strategy  will  be  discussed  In  the  following  section. 
But,  It  Is  important  that  the  reader  understand  dual 
sourcing  along  with  the  following  criticisms  and  their 
possible  effect  on  cost  savings  estimates. 
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Non-recurring  Coats .  The  non-recurring  or  one-time 
costs  to  develop  a  second  source  for  production  can  be 
extensive.  These  costs  can  include  technology  transfer, 
purchasing  re-procurement  data,  special  tooling  and  test 
equipment,  and  second  source  qualification.  Data  available 
from  studies  on  missile  programs  cite  non-recurring  costs  in 
the  range  of  $20  million  to  $100  million  (Fy84)  (12:3-18). 
k  1985  estimate  of  the  total  cost  to  establish  a  second 
source  for  C-17  production  was  $2.3  billion  in  FY81  dollars 
(15:1).  However,  these  investments  are  made  in  anticipation 
that  out  year  recurring  costs  will  be  reduced,  offsetting 
the  initial  investment.  Navy  officials  estimate  that 
because  savings  from  a  second  source  normally  range  between 
15  and  30  percent,  the  service  recovers  the  non-recurring 
cost  of  going  to  a  second  source  in  less  than  two  years  in 
most  oases  (11:43) . 

Contractor  Price  Gaming.  In  a  dual  source  competition, 
the  lower  price  bidder  usually  is  awarded  the  larger  share 
of  the  government' s  annual  requirements  while  the  higher 
priced  bidder  gets  a  smaller  share.  This  smaller  share  is 
at  least  a  minimally  sustaining  quantity  to  keep  the 
production  line  open.  The  possibility  of  the  whole  award 
going  to  the  lowest  price  bidder  would  undermine  competition 
in  future  years  and  essentially  waste  the  government's 
investment  in  developing  the  second  source.  After  the 
government  spends  considerable  money  developing  a  second 
source,  they  are  not  likely  to  let  that  source  evaporate  by 
not  utilizing  the  production  capacity.  Competition 
advocates  concede  that  once  they  have  paid  to  bring  a  second 
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•ouratt  Into  a  program,  dropping  althar  company  altogether 
rarely  makes  economic  sense  (11:44).  Therefore,  the 
situation  of  an  assured  level  of  production  allows  for  price 
gaming  where  contractors  have  the  opportunity  to  submit 
inflated  bid  prices.  Recalling  when  a  contract  is 
oategorlzed  as  competitive,  detailed  cost  and  pricing  data 
is  not  required,  and  seldom  obtained.  Inflated  bid  prices 
oan  be  submitted  by  both  the  lower  and  higher  priced 
contractors.  The  lower  priced  contractor,  normally  the 
initial  contractor,  will  not  feel  competitive  pressure  from 
a  recently  developed  second  source.  The  second  source 
contractor  will  likewise  not  feel  competitive  pressure  and, 
being  assured  a  minimally  sustaining  production  quantity, 
bid  inflated  prices  content  to  be  a  ’’happy  loser”  (3:14). 

Referring  again  to  the  IQ  report,  IG  officials  believe 
that  guaranteed  oontraots  for  the  smaller  portion  of  a  dual 
source  award  should  be  classified  as  non-oompetitlve 
procurements  using  oompetltlve  procedures  (11:44). 
Classifying  these  awards  in  this  manner  would  allow  closer 
examination  of  contractor  cost  and  pricing  data,  therefore 
minimizing  the  potential  for  contractor  price  gaming. 

Lost  Economic  Beneflta .  Loss  of  economic  benefits  fall 
into  two  categories:  reduced  learning  curve  and  diseconomies 
of  scale.  In  a  dual  source  program,  production  quantities 
must  be  split,  in  some  ratio,  between  two  or  more 
contractors.  This  splitting  of  production  quantities  does 
not  allow  either  contractor  to  fully  realize  savings  from 
the  learning  curve  effect.  Also,  the  split  production  most 
likely  will  cause  unused  production  capacity  in  both 
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oontraotor's  facilities,  not  allowing  the  full  realization 
of  ooonomlea  of  scale.  In  both  cases,  the  result  should  be 
higher  unit  production  costs  than  if  the  entire  quantity 
were  awarded  to  the  low  bidder.  Again,  the  argument  for 
competition  is  that  the  reduced  profit  margins  bid  by  the 
oontraotors  will  more  than  offset  the  higher  unit  coats 
resulting  from  reduced  learning  curve  effects  and 
diseconomies  of  scale. 

Industry  Capacity.  When  the  defense  industry  has 
considerable  idle  capacity,  contractors  are  more  likely  to 
bid  lower  prices  as  a  result  of  increased  competition  for 
limited  work.  On  the  other  hand,  when  there  are  numerous 
contracting  opportunities,  contractors  face  less  pressure  to 
bid  competitively,  and  submit  higher  bid  prices.  Therefore, 
there  is  a  correlation  between  the  capacity  of  the  defense 
industrial  base  and  a  contractor's  bid  price. 

ggiigg  Studita 

The  arguments  for  increased  competition,  which 
culminated  in  the  Competition  in  Contracting  Act,  were  baaed 
on  the  belief  that  competition  saves  time,  reduces  risk, 
improves  performance,  and  saves  money.  However,  the  prior 
studies  that  led  up  to  CICA  provided  quantitative  proof  that 
competition  saves  money,  were  limited  in  scope  and  depth. 

In  order  to  quantify  the  effects  of  competition,  these 
studies  focused  on  dual  source  acquisitions  that  were 
previously  sole  source,  and  then  only  measured  the  estimated 
dollar  savings  at  contract  award.  Figure  2  shows  the  price 
improvement  curve  and  estimated  savings  from  competition  and 
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dapiots  th«  methodology  used  In  th«i  prior  studies  for 
determining  oompetltlon' s  savings. 


Figure  2)  Priaa  Improvemsnt  Cutvs  (2t6) 


Recall/  this  methodology  was  developed  for  sole  source 
programs  that  were  subsequently  competed  with  dual  sourcing. 
The  initial  phase  of  the  learning  curve  was  developed  with 
actual  sole  source  unit  cost  data.  From  the  point  where 
competition  is  introduced/  the  sole  source  learning  curve  is 
extrapolated  and  compared  to  actual  competed  unit  costs, 
points  A,  B/  and  C.  The  difference  is  the  realized  cost 
savings  resulting  from  competition. 

Bven  with  this  fundamental  relationship  to  define  the 
cost  savings  from  competition,  the  dual  source  criticisms 
mentioned  earlier  oast  doubt  on  the  validity  of 
competition's  "savings"  claim.  In  1984,  Lieutenant  Colonel 
Richard  J.  Hampton,  USAF,  a  Research  Fellow  at  the  Airpower 
Research  Institute,  did  an  extensive  review  of  8  previous 
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studies  encompassing  45  previously  sole  source  systems  that 
were  since  competed  using  dual  source  procurement 
(10:41-83) .  He  reported  estimated  percentage  savings  rates 
from  11.8%  to  53.3%.  However,  he  also  reported  some  serious 
drawbacks  in  eaoh  of  the  studies. 

The  foremost  drawback,  oommon  among  all  studies,  was 
the  lack  of  consideration  of  the  time  value  of  money.  The 
outyear  competitive  savings  were  not  discounted  to  a  oommon 
base  year  and,  as  such,  should  not  have  been  simply 
subtracted  from  up-front  non-recurring  costs.  To  make 
matters  worse,  seven  of  the  studies  never  figured  any  non¬ 
recurring  coats  into  their  evaluation,  while  the  other  only 
considered  some  non-recurring  costs.  Quoting  Hampton, 

These  figures  do  not  represent  the  net  impact  to 
the  government  of  dual  sourcing.  Taken  together, 
these  concerns  oast  a  shadow  of  caution  on  the  use 
of  any  rule-of-thumb  estimate  of,  for  example,  a 
10%,  25%,  or  other  fixed  percentage  savings 
available  when  systems  are  competed.  (10:61) 

Still,  blanket  estimates  of  the  potential  savings  from 

competition  are  cited.  In  a  1985  Air  Force  System  Command 

review  of  the  C-17  second  sourcing  analysis,  officials 

claimed  "When  competition  is  introduced,  a  25%  decrease  in 

unit  price  is  assumed  for  both  sources"  (15:1). 

Recalling,  the  prior  studies  were  based  on  dual  source 

programs.  As  Hampton  discovered,  these  prior  studies  had 

some  serious  methodological  drawbacks.  In  addition,  dual 

sourcing  has  some  valid  criticisms  as  mentioned  earlier. 

Still,  the  prior  studies,  the  results  of  which  became  the 

standard  estimate  for  the  savings  attributable  to 

competition,  were  limited  in  scope  and  depth.  This  research 
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will  extend  the  study  of  the  purported  savings  attributable 
to  oompetition  by  expanding  the  prior  studies  in  two  areas. 

This  research  will  expand  the  scope  of  the  prior 
studies  by  examining  competitive  versus  sole  source  contract 
awards  and  analyzing  the  cost  and  schedule  growths  of  each. 
The  prior  studies  were  limited  in  scope  because  they  only 
looked  at  previously  sole  source  programs  that  were 
subsequently  competed  with  a  dual  source  strategy.  A  large 
portion  of  the  cost  for  weapon  systems  the  DoD  procures  each 
year  are  of  a  high  value,  limited  quantity  nature,  that  do 
not  apply  to  annual  quantity  buys.  "Over  50%  of  the  dollars 
spent  by  Air  Force  Systems  Command  during  the  first  half  of 
the  1980'a  went  to  only  four  major  systems"  (8:144). 
Estimating  a  25%  savings  due  to  competition  on  a  dual  source 
tactical  missile  program,  then  applying  that  same  25% 
savings  estimate  to  a  C-17  program,  is  a  fallacy  in 
reasoning,  and  assumes  the  two  programs  are  comparable, 
simply  because  they  were  both  competed. 

This  research  will  also  expand  on  the  depth  of  the 
prior  studies  by  examining  the  cost  and  schedule  behavior 
over  the  life  of  the  contract,  as  opposed  to  simply 
estimating  savings  at  contract  award. 

The  expanded  scope  and  depth  of  this  research  should 
make  the  results  more  applicable  to  Air  Force  managers. 

Conclusion 

Blanket  cost  savings  estimates  continue  to  be  made 
based  on  the  philosophy  of  the  price  improvement  curve  In 
Figure  2,  and  the  methodology  of  comparing  system  unit  costs 
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on  dual  source  contracts  that  were  previously  sole  source. 
Btneflta  from  competition  may  Indeed  exist  on  large 
programs,  but  applying  savings  estimates  that  were  based 
upon  dual  source  programs  Is  misleading. 

Chapter  III  details  the  methodology  used  to  answer  the 
research  questions  and  meet  the  research  objectives:  Is 
there  a  difference  In  cost  and  schedule  growth  between 
competed  and  sole  source  contracts? 
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III. 


gYirYiiw 

Th«  methodology  employed  in  this  research  used  data 
from  the  Acquisition  Management  Information  system 
maintained  by  Aeronautical  Systems  Center' s  Directorate  of 
Contracting  ASC/PK.  Operational  definitions  of  key  terms  or 
concepts  are  required  in  order  to  accurately  address  the 
investigative  questions.  These  terms  and  concepts,  as  well 
as  the  methodology  used  to  define  these  concepts,  are 
covered.  Discussion  then  focuses  on  the  population  of 
interest  and  data  collection  efforts.  Also,  assumptions  and 
limitations  of  the  analysis  are  identified.  Finally,  the 
statistical  tests  conducted  on  the  research  hypotheses  are 
presented. 

Research  Design 

This  research  was  designed  to  answer  the  three  research 
questions  described  in  Chapter  I.  The  research  questions 
and  the  hypotheses  tested  are  listed  below. 

Research  Question  #1.  For  cost  growth  and  schedule 
growth  categories,  does  the  mean  growth  for  a  competed 
contract  differ  from  the  mean  growth  of  a  sole  source 
contract? 

Hypothesis . 

Hot  Mean  Competed  Growth  -  Mean  Sole  Source  Growth 

Hat  Mean  Competed  Growth  *  Mean  Sole  Source  Growth 
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Research  Question  #2.  For  the  cost  growth  category 
only,  does  the  mean  cost  growth  for  competed  and  sole  source 
lower  risk  contracts  differ  from  competed  and  sole  source 
higher  risk  contracts? 

Hoi  LOW  Risk  Moan  Growth  •  High  Risk  M&mn  Growth 
Ha  I  Low  Risk  Moan  Growth  ^  High  Risk  Maan  Growth 

Research  Question  #3.  For  the  cost  growth  category 
only,  does  the  mean  cost  growth  for  competed  and  sole  source 
lower  price  contracts  differ  from  competed  and  sola  source 
higher  price  oontracts? 

HYBQthftAi.a.L, 

Hoi  LOW  Prica  Maan  Growth  •  High  Prica  Maan  Growth 

Ha  I  Low  Prica  Maan  Growth  High  Prica  Maan  Growth 


The  research  relied  on  data  from  the  Air  Force's 
Acquisition  Management  Information  System  (AMIS) .  In 
particular,  this  study  used  the  Program  Management  System 
(PMS)  and  CONTRACT  data  bases.  The  PMS  and  CONTRACT  data 
bases  were  used  to  capture  coat  and  schedule  Information 
respectively,  for  the  period  of  January  1980  to  December 
1990.  This  eleven  year  period  was  used  to  mitigate  bias  in 
this  study  due  to  fluctuations  In  the  defense  business 
cycle.  Also,  It  provides  for  5  years  of  pre  CICA  and  6 
years  post  CICA  awards.  Specifically,  the  PMS  data  base  was 
used  to  capture;  Initial  contract  price,  final  or  current 


28 


contract  price,  date  of  award,  type  contract,  number  of 

respondents,  solicitation  procedures,  and  the  extent 

competed.  To  answer  the  research  question  concerning  cost 

growth,  data  was  collected  on  contracts  managed  by  the 

Wright  Aeronautical  Laboratories  as  well  as  the  following 

Product  Centers  in  Air  Force  Material  Command: 

Aeronautical  Systems  Center 
Electronic  Systems  Center 
Space  Systems  Center 
Armament  Systems  Center 

The  population  of  contracts  used  to  measure  schedule 
growth  differed  somewhat  from  that  used  to  measure  cost 
growth.  Schedule  growth  data  was  collected  solely  from 
contracts  administered  by  the  Air  Force  Contract  Management 
Division  (AFCMD)  at  Kirtland  AFB,  NM.  This  population  was 
used  because  AFCMC  uses  the  AMIS  system  to  record  contract 
deliveries  for  payment  purposes.  Other  data  bases  were 
available  but  the  accuracy  and  validity  of  the  data  could 
not  be  assured  as  readily  as  data  from  the  AMIS  system. 

Research  Population 

Cost  Growth.  The  population  of  contracts  for  cost 
growth  was  establish  by  first  identifying  all  contracts 
awarded  at  the  five  organizations  listed  earlier  between 
1980  and  1990.  Then,  all  contracts  having  an  initial 
contract  price  of  $0  were  eliminated  from  the  population. 
These  contracts  were  eliminated  because  an  initial  award  of 
$0  would  not  allow  a  cost  growth  to  be  calculated.  Also,  it 
is  assumed  that  these  contracts  were  structured  with  a 
series  of  preplanned  options  which  do  not  truly  represent 
cost  growth.  Next,  contracts  identified  in  the  PMS  data 
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base  field  "Solicitation  Procedures”  as  basic  research,  time 
and  material,  or  labor  hour  contracts  were  eliminated. 

Also,  contracts  described  as  providing  services  were 
eliminated  because  they  are  not  representative  of  the  weapon 
systems  procurements  of  interest  in  this  study.  Finally, 
contracts  identified  in  the  "Type-Contract"  field  as  being  a 
combination  coat  and  fixed  priced  line  items  were  eliminated 
because  they  did  not  fit  into  the  risk  categories  defined 
for  this  study. 

Schedule  Growth.  First,  all  contracts  administered  by 
AFCMO  with  an  initial  award  date  between  1980  and  1990  were 
identified.  Then,  contracts  identified  in  the  PMS  Contract 
Data  Base  field  "Solicitation  Procedures"  as  basic  research 
or  time  and  material  type  contracts  were  eliminated.  These 
contracts  were  eliminated  because  the  scope  of  this  research 
was  weapon  syatsm  development  and  production  contracts. 

Contract  Classification 

In  order  to  answer  the  research  questions  and  to  test 
the  hypotheses,  classification  of  the  contracts  into  three 
categories  was  required.  These  three  categories  are  either 
sole  source  or  competed,  risk  (higher  or  lower) ,  and  initial 
price  (higher  or  lower) .  The  methodology  used  to  make  these 
classifications  is  now  explained. 

Sole  Source /Competed.  The  decision  process  that 
classified  contracts  into  sole  source  and  competed 
categories  used  the  PMS  data  base  fields  "Extent  Competed" 
and  "Number  Received".  Figure  3  illustrates  the  decision 
process  used. 
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Thtt  "Extent  Comp«t«d"  column  In  tho  PMS  data  base  haa 
four  poaaiblo  claaslfioations : 

A:  Competad 

B:  Not  Available  for  Competition 

C:  Follow-on  to  Competition 

D :  Not  Competed 

The  "Number  Received"  field  in  the  PMS  data  base  has 
three  possible  entries;  "0",  "1"  or  "2".  The  "Extent 
Competed"  field  was  used  with  the  field  "Number  Received"  to 
categorize  each  contract  as  competed  or  sole  source.  A  "1" 
entry  means  only  one  proposal  was  received  in  response  to 
the  governments  solicitation  while  a  "2"  entry  means  that 
two  or  more  proposals  were  received  in  response  to  the 
governments  solicitation.  The  "0"  entry  identified  in 
Figure  3  is  required  because  this  field  was  not  available  on 
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contracts  awarded  prior  to  1385.  Contracts  issued  prior  to 
1985  are  thus  aategorized  solely  on  the  Information 
contained  in  the  "Extent  Competed"  field.  An  example  helps 
to  illustrate  the  logic  for  classifying  a  contraot  as 
competed  or  sole  source.  If  a  oontraot  lists  an  "A"  in  the 
"Extent  Competed"  category  but  shows  only  one  response  to 
the  governments  solicitation,  this  research  defines  the 
oontraot  as  sole  source.  The  logic  behind  this 
categorization  la  that  competition  does  not  exist  when  only 
one  source  answers  the  Request  for  Proposal.  In  order  to 
receive  the  benefits  of  competition,  at  least  two  qualified 
sources  must  be  available. 

Risk  categorization  is  required  for  the  tests  on 
cost  growth.  The  risk  assigned  to  each  contract  is 
identified  by  the  "type-oontract"  field  in  the  PMS  contract 
data  base.  A  letter  designation  in  the  field  identifies  the 
specif io  type  of  oontraot  used  for  the  effort.  This 
research  operationally  defines  all  "cost"  type  contracts  as 
higher  risk  while  defining  all  "fixed  price"  type  contracts 
as  lower  risk.  These  definitions  are  generally  excepted  by 
Air  Force  contracting  officers. 

Initial  Price.  Contracts  were  divided  into  higher  and 
lower  initial  price  categories  by  selecting  the  median 
contract  price  as  the  dividing  line.  This  resulted  in  two 
equally  sized  samples  for  lower  and  higher  price  contracts. 

RflAlgm  SarnpUT^q 

Cost  Growth.  Contracts  from  each  of  the  four  Product 
Centers  and  Wright  Aeronautical  Labs  were  categorized  into 
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either  competed  or  sole  source  and  higher  or  lower  risk. 
After  categorizing  the  population  all  contracts  were 
numbered  in  each  category  and  a  random  number  generator  was 
used  to  select  a  sample  of  contracts  from  the  population  in 
each  category.  The  sample  size  selected  from  each  Product 
Center  and  the  Labs  were  determined  by  the  relative 
percentage  of  the  total  populati  va  that  a  Product  Center  or 
the  Labs  were  responsible  for.  Table  1  lists  the  number  of 
contracts  the  Product  Centers  and  the  Labs  contributed  to 
the  overall  population  in  each  of  the  categories;  competed 
or  sole  source  and  lower  and  higher  risk. 


TABLE  1 

CONTRACTS  CONTRIBUTING  TO  THE 
OVERALL  POPULATION 


COMP 

ETED  SOLE 

SOURCE 

Low  Risk 

High  Risk 

High  Risk 

Low  Risk 

Space  Systems 
Center 

78 

37 

36 

30 

Armament 
Systems  Center 

189 

77 

81 

35 

Aeronautical 
Systems  Center 

174 

21 

226 

20 

Electronic 
Systems  Center 

168 

186 

81 

90 

Wright  Labs 

315 

997 

124 

537 

Sub-Total 

924 

1318 

548 

712 

TOTAL 

2242  1260 

The  Table  2  lists  the  population  sizes  N,  the  aub" 
totals  from  Table  1,  and  the  size  of  the  random  sample  n 
drawn  in  each  category  to  ensure  the  desired  level  of 
accuracy. 
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TABLE  2 

POPULATION/SAMPLB  SIZES 


Lower  Risk 

Higher  Risk 

Competed 

N  -  924 

N  -  1318 

n  -  137 

n  «  145 

Sole 

N  -  548 

N  -  712 

Source 

n  -  122 

n  -  130 

An  Initial  aampla  of  30  contraota  from  each  category 
le.  Competed/ Lower  Risk  was  drawn,  and  estimates  of  the 
population  mean,  standard  deviation  and  standard  error  were 
determined.  These  initial  estimates  were  used  to  calculate 
the  sample  size  required  for  each  category  at  a  significance 
level  equal  to  .05  using  the  following  formula  described  by 
Soheaffer  (12:69). 

rt-  - ^ - 

(tr-  i)D  *  o» 
where:  d  •  and 

B  •  2  X  Standard  error 
of  the  estimator 

Using  the  value  of  2  for  the  definition  of  "B"  results 
in  a  better  than  95%  confidence,  that  with  the  sample  size 
n,  the  true  mean  cost  growth  is  within  2  standard  errors  of 
the  estimated  mean  (12:68). 

Schedule  Growth.  After  categorizing  the  schedule 
growth  population  relative  to  the  level  of  competition,  the 
contracts  in  each  category  were  numbered,  and  a  random 
number  generator  was  used  to  select  a  sample  of  contracts 
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from  the  population  In  each  category.  The  population  sizes 
for  sole  souroe  and  competitive  contracts  were  198  and  572 
respectively.  An  sample  was  taken  for  each  category  and  an 
estimate  of  the  population  mean,  standard  deviation  and 
standard  error  were  determined.  The  initial  estimates  were 
used  to  oaloulate  the  sample  sizes  required  for  competed  and 
sole  souroe  categories  using  Soheaffers'  formula  referenced 
earlier.  The  resulting  sample  sizes  were  n  ■■  18  for  sole 
souroe  contracts  and  n  -  20  for  competed  contracts. 

Operational  Definitions 

Five  terms  or  conoepts  must  be  defined  for  each 
contract  in  the  research  sample  to  answer  the  research 
questions  and  test  the  hypotheses.  The  purpose  of  these 
definitions  are  to  support  the  calculation  of  the  cost  and 
schedule  growth  for  each  individual  contract. 

Award  Date.  The  contract  award  date  information  is 
required  to  compute  the  schedule  growth  for  the  contract. 

The  PMS  data  base  field  identifying  the  contract  "Date- 
Distribution"  is  used  as  the  baseline  contract  award  data 
for  all  calculations.  This  information  also  allowed 
determination  of  the  relative  contract  age. 

Initial  Contract  Schedule.  The  contract  delivery 
requirement  for  a  contract  line  item  on  the  original  basic 
contract.  The  field  in  the  CONTBACT  data  base  defining  this 
variable  is  "Scheduled  Delivery". 

Final  Schedule.  The  actual  delivery  date  for  a 
contract  line  item.  The  field  in  the  CONTRACT  data  base 
defining  this  variable  is  "Delivery  Date". 
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Initial  Target  Price.  This  is  the  initial  price  for 
the  basic  contract.  This  data  is  available  in  the  PMS 
Contract  Data  Base  and  is  required  for  the  calculation  of 
cost  growth.  The  field  in  the  PMS  data  base  defining  this 
variable  is  "Tot-Amount" . 

Currant  Target  Price .  This  is  the  current  price  of  the 
contract  and  includes  the  rolled  up  price  of  the  basic 
contract  and  all  modifications  through  15  June  1993.  This 
data  is  available  in  the  PMS  data  base  and  is  used  with  the 
initial  target  price  to  calculate  cost  growth.  The  field  in 
the  data  base  defining  this  variable  is  "Contract  Price". 

Research  Variables 

Schedule  Orowth.  Schedule  growth  was  measured  using 
schedule  growth  factors  which  are  defined  as  the  difference 
in  the  completed  date  of  a  contract  line  item  and  the  line 
item's  initially  scheduled  date  divided  by  the  initial 
schedule  length.  A  schedule  growth  factor  is  therefore  the 
percentage  increase  or  decrease  in  the  delivery  of  the  line 
item.  This  increase  or  decrease  was  determined  using  the 
last  completed  line  item  on  the  contract  to  obtain  the  most 
representative  schedule  performance.  Only  line  items 
established  with  the  basic  contract  award  were  used  in  the 
analysis  because  this  was  considered  more  representative  of 
the  true  schedule  as  opposed  to  a  line  item  established  with 
a  later  contract  modification.  The  PMS  Contract  Data  Base 
field  "Date-Distribution"  allowed  calculation  of  the  initial 
and  final  schedules.  The  "Date-Distribution"  date  was 
assumed  to  be  the  contract  award  date. 
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To  establish  the  schedule  growth  on  a  contract,  the 
last  basic  line  item  was  used  to  establish  the  schedule 
growth  of  the  sample  contract.  This  methodology  was  used  to 
capture  a  more  realistic  schedule  performance  on  the 
oontraot.  The  logic  of  this  methodology  is  that  any 
sohedule  problems  will  be  magnified  with  time. 

A  possible  problem  with  this  methodology  is  that 
considerable  schedule  growth  may  be  attributable  to  "getting 
the  paperwork  in  order"  prior  to  contract  close-out. 

Schedule  growth  could  be  greater  in  this  case,  however  the 
increases  should  be  equivalent  for  the  competed  and  sole 
source  populations.  Another  possible  problem  with  this 
methodology  is  that  the  oontraot  may  have  experienced 
increased  schedule  growth  since  the  last  milestone  was 
completed,  yet  that  information  was  not  captured.  When 
using  active  contracts  to  compute  cost  or  sohedule  growth 
parameters,  the  growth  calculated  may  not  be  indicative  of 
the  cost/sohedule  performance  for  a  completed  effort  because 
the  last  5%  of  an  effort  usually  is  the  slowest  part  of  a 
program.  These  effects  were  monitored  by  tracking  the  mean 
age  of  the  contracts  included  in  the  study. 

Cost  Growth.  Cost  growth  was  measured  using  cost 
growth  factors  which  are  defined  as  the  difference  between 
the  final,  or  latest,  contract  price  and  the  initial 
contract  price,  divided  by  the  initial  price.  The  cost 
growth  factor  is  therefore  the  percentage  increase  or 
decrease  of  the  contract  price.  The  methodology  to 
determine  the  cost  growth  for  completed  and  on-going 
contracts  was  the  same.  Similar  to  schedule  growth,  a 
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possible  problem  with  this  methodology  is  that  on-going 
oontraots  may  exhibit  a  smaller  coat  growth.  These  effects 
were  again  monitored  by  tracking  the  mean  age  of  the 
oontraots  Included  In  the  sample. 

Ajgmliigng 

It  was  recognized  that  several  factors  Influence  every 
program's  cost  and  schedule  performance.  An  attempt  to 
quantify  the  effects  of  these  factors  would  be  Impossible. 
Below  is  a  list  of  events  that  can  affect  the  cost  and 
schedule  performance  for  both  sole  source  and  competed 
oontraots, 

Changing  Requirements  (Specification,  Schedule) 
Delays/Cost  increases  due  to  funding  outs 
Delays  due  to  late  GFB  (hardware,  test  facilities) 
Effects  of  follow  on  contract  award  potential 
Economic  conditions 
Political  considerations 

By  using  a  statistically  significant  sample  of  the 
contract  populations,  these  effects  are  assumed  to  equally 
affect  sole  source  and  competed  contracts. 

Limitations 

This  research  was  limited  by  the  data  available  through 
the  AMIS  system.  Data  fields  are  defined  in  AMIS 
documentation  but  are  subject  to  the  users'  interpretation. 
Also,  using  data  base  information  subjects  the  research  to 
input  errors  as  well  as  omission  of  cost  and  schedule 
updates.  The  AMIS  system  was  used  in  place  of  original 
contract  documents  due  to  the  time  constraints  on  the 
research  effort.  Using  a  data  base  allowed  the  researchers 
to  obtain  contract  cost  and  schedule  information  on  numerous 
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contracts  very  quickly.  Also,  this  allowed  the  researchers 
to  include  sample  contracts  from  locations  all  over  the 
country.  This  provided  a  sample  population  representative 
of  Air  Force  Material  Command  procurements.  Finally,  this 
research  methodology  was  limited  in  scope  to  contracts 
awarded  prior  to  1  January  1990.  This  limitation  requiring 
contracts  to  be  active  for  at  least  3  years  allowed  those 
on-going  contracts  enough  time  to  begin  to  exhibit  some 
schedule  and  cost  growth  that  may  not  be  apparent  on  newer 
contracts.  On-going  contracts  were  used  because  their 
exclusion  would  have  limited  the  sample  size  of  the  study. 
Also,  it  is  common  for  DoD  contracts  to  remain  open  for  many 
years.  These  contracts  are  viable  for  this  research  since 
it  attempts  to  measure  growth  rates.  Older  open  contracts 
are  likely  to  establish  the  true  growth  trends  of  the 
oontract  thus  are  suitable  for  this  research. 

Statistical  Analysis 

Statistical  analysis  was  conducted  to  test  the 

hypotheses  and  answer  the  research  questions.  Before 

conducting  the  statistical  analysis,  the  sample  size  in  each 

category  of  interest  was  trimmed  by  10%.  This  trimming  was 

accomplished  by  deleting  5%  of  the  cost  growth  factors  from 

the  low  and  high  extremes  of  the  frequency  distribution. 

Using  a  trimmed  mean  with  a  moderate  trimming 
proportion  will  yield  a  measure  that  is  neither  as 
sensitive  to  outliers  as  the  mean  (since  any  small 
number  of  outliers  will  be  deleted  before 
averaging)  nor  as  insensitive  as  the  median . (7 ; 19) 

Because  the  research  questions  differed  between  the 

cost  growth  analysis  and  the  schedule  growth  analysis,  each 
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analysis  will  be  discussed  separately,  with  the  analysis  of 
the  cost  growth  disoussed  first. 

Coat  Growth.  Eaoh  of  the  research  questions  contained 
a  hypothesis  that  will  be  tested  in  the  statistical 
analysis.  The  analysis  of  Researoh  Question  #1  was 
Qonduoted  in  three  phases  as  shown  in  Figures  4,  5,  and  6 
respectively.  The  analysis  of  Research  Question  #2  was 
completed  using  two  phases,  as  depicted  in  Figures  5  and  6. 
The  analysis  of  Research  Question  #3  was  completed  in  only 
one  phase,  as  depicted  in  Figure  6. 

Teats  of  Research  Question  *1.  The  first  research 
question  was  tested  in  all  three  phases  as  showing  in 
Figures  4,  5,  and  6.  The  first  phase.  Figure  4,  tested  for 
differences  in  the  mean  cost  growth  between  competed 
contracts  and  sole  source  contracts.  The  second  phase, 
testing  vertioally  in  Figure  5,  tested  for  differences  in 
the  mean  cost  growth  between  lower /higher  rislc  competed 
contracts  and  lower/higher  risk  sole  source  contracts.  This 
phase  included  2  tests.  The  third  phase,  testing  vertically 
in  Figure  6,  tested  for  differences  in  the  mean  cost  growth 
between  corresponding  categories  of  risk  and  initial  price 
for  competed  and  sole  source  contracts.  For  example,  one 
test  compared  the  mean  cost  growth  between  lower  risk/lower 
price  competed  contracts  and  lower  risk/lowor  price  sole 
source  contracts.  This  phase  included  4  tests. 

Testa  of  Research  Question  *2.  The  second 
research  question  was  tested  in  2  phases  as  shown  in  Figures 
5  and  6.  The  first  phase,  testing  horizontally  in  Figure  5, 
tested  for  differences  in  the  mean  cost  growth  between 
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competed/ sole  source  lower  risk  contracts  and  competed/sole 
source  higher  risk  contracts.  This  phase  involved  2  tests. 
The  second  phase,  testing  horizontally. in  Figure  €,  tested 
for  differences  in  the  mean  cost  growth  between  lower  and 
higher  risk  oontracts,  within  competed/ sole  source  and 
hlgher/lcwer  initial  price  categories.  For  example,  one 
test  compared  the  mean  cost  growth  between  competed/ lower 
price,  lower  risk  contracts  and  competed/ lower  price,  higher 
risk  contracts.  This  phase  involved  4  tests. 

Tests  for  Research  Question  #3.  The  third 
research  question  was  tested  in  only  1  phase.  This  phase, 
testing  vertically  In  Figure  6,  tested  for  differences  in 
the  mean  cost  growth  between  lower  and  higher  initial  price 
contracts,  within  oompeted/sole  source  and  lower/higher  risk 
categories.  For  example,  one  teat  compared  the  mean  cost 
growth  between  competed/ lower  risk,  lower  price  contracts 
and  competed/ lower  risk,  higher  price  contracts.  This  phase 
involved  4  tests. 


Competed 

Sole  Source 

Mean  Cost  Growth 

Mean  Cost  Growth 

fi'gure' . ’4"  ^"TIJaTe  1  Statistical  (Uests 


Lower  Risk 

Higher  Risk 

Competed 

Mean  Cost  Growth 

Mean  Cost  Growth 

Sole  Source 

Mean  Cost  Growth 

Mean  Coat  Growth 
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An  lnt«nn«diate  st«p  prior  to  tasting  the  hypothesis 
was  to  establish  a  oonfldence  interval  estimation  of  the 
true  mean. 

Confidence  Interval  Estimation.  Because  the 
sample  sizes  are  large «  the  large  sample  confidence  Interval 
for  the  population  mean  is  defined  by  Devore  (6:267)  as: 

For  this  analysis,  a  95%  confidence  interval  was  used 
in  all  cases.  The  confidence  Interval  can  be  interpreted  to 
mean,  if  the  experiment  were  repeated,  95%  of  the  time  the 
true  mean  cost  growth  would  fall  within  the  computed 
confidence  Interval. 

Test  for  Difference  Between  Population  Means.  The 
statistical  analysis  will  also  test  for  a  difference  in  the 
population  means,  using  the  sample  mean  as  an  estimator  of 
the  population  mean.  Because  the  sample  sizes  are  large, 
the  large  sample  test  statistic,  as  presented  by  Devore 
(6:332)  is  appropriate  for  these  tests. 
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Large  Sample  Test  Statistic: 


z 


X  -  y  -  Ao 


where  Aq  ■  0  when  is  true 


Here,  m  «  the  sola  source  sample  size  and  n  »  the 
competed  sample  size.  A  two-tailed  test  was  conducted  for 
each  of  the  hypotheses  at  a  significance  level  equal  to  .05. 
The  decision  rule  stated:  If  the  test  statistic  is  either 
greater  than  z^rit  less  than  -Zatit  null  hypothesis  (H,) 
is  rejected,  and  the  difference  in  population  means  is 
statistically  significant.  Here,  is  defined  as: 

^erit  ® 

2 


Schedule  Growth.  The  analysis  for  schedule  growth  will 
be  more  simplified  that  the  cost  growth  analysis.  Because 
the  schedule  growth  data  is  not  aggregated  by  level  of  risk 
or  initial  price,  the  statistical  analysis  will  only  consist 
of  a  test  for  the  difference  between  population  means. 

Test  for  Difference  Between  Population  Means. 

This  analysis  is  vastly  different  from  the  analysis  of  the 
difference  between  population  means  conducted  for  the  cost 
growth  data  because  the  sample  sizes  used  for  both  competed 
and  sole  source  contracts  are  not  sufficient  to  assume 
normality  in  the  test  statistic.  In  this  case,  the  Wilcoxon 
Rank-Sum  Test,  is  the  appropriate  test  (6:611).  The 
Wilcoxon  Rank-Sum  teat  requires  respective  samples  to  be 
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labeled  such  that,  the  sample  sizes  are  ordered  where  m  is 
less  than  or  equal  to  n.  For  this  analysis,  the  sample  size 
of  the  sole  source  data  is  m  -  18,  and  the  competed  data  is 
n  -  20. 

After  combining  and  ordering  the  schedule  growth 
factors  for  both  competed  and  sole  source  contracts,  the 
test  statistic,  w,  is  defined  as  the  sum  of  the  ranks  of  the 
sole  source  contracts  (7:612).  According  to  Devore,  when 
both  m  and  n  are  greater  than  8,  the  distribution  of  P/  can 
be  approximated  by  an  appropriate  normal  curve  (7:613).  To 
obtain  the  approximation,  the  mean  and  variance  of  must  be 
determined  and  are  defined  as  follows  (7:613). 

Mean  of  N': 

r  fcf 

Variance  of  P/: 

_  mn(jn+n+l) 

<»..  -  - 12 - 

A  Central  Limit  Theorem  can  then  be  used  to  conclude 
that  when  H,  is  true,  the  test  statistic,  Z,  has 
approximately  a  standard  normal  distribution.  The  test 
statistic  Z  is  defined  as  (7:613): 


This  test  statistic  was  then  tested  using  a  two-tailed 
test  with  a  significance  level  equal  to  .05.  The  decision 
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rule  stated:  If  the  test  statistic  is  either  greater  than 
Suit  or  less  than  the  null  hypothesis,  H,  is  rejected, 

and  the  difference  in  population  means  is  statistically 
significant.  The  value  of  Zont  is  the  same  as  defined 
earlier. 

Contract  Aae.  Analysis  of  the  sample's  mean  contract 
age  was  not  conducted  however,  the  determination  of  the  mean 
age  allowed  monitoring  of  this  variable  and  its  possible 
effects  on  the  research  variables  cost  growth  and  schedule 
growth . 

Conclusions 

The  AMIS  system  allowed  the  researchers  to  access  a 
representative  cross  section  of  Air  Force  Material  Command 
contracts.  Also,  the  data  fields  in  the  PMS  and  CONTRACT 
data  bases  allowed  the  researchers  to  categorize  sole  source 
and  competed  contracts  and  to  further  stratify  these 
contracts  relative  to  risk  and  initial  contract  price. 
Breaking  down  the  population  relative  to  the  extent  of 
completion,  risk  and  initial  price  supports  the  hypothesis 
testing  required  to  answer  the  research  questions  of  this 
thesis. 
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IV. 


The  findings  and  analysis  are  presented  in  two 
sect ions.  The  first  seotion  presents  the  raw  data  in  the 
form  of  histograms;  sample  statistics;  and  the  confidence 
interval  estimations . 

The  second  section  presents  the  results  of  the 
statistical  analysis  on  the  research  hypothesis  and  the 
research  questions.  The  results  are  presented  in  an  order 
corresponding  to  the  research  questions  ie.,  tests  between 
competed  and  sole  source  contracts;  tests  between  lower  and 
higher  risk  contracts;  and  finally  tests  between  lower  and 
higher  initial  price  contracts. 

RlM.,BAtaL 

The  raw  data  consists  of:  histograms;  sample 
statistics;  and  confidence  interval  estimations.  The  cost 
growth  raw  data  is  presented  first,  with  the  data  for 
competed  contracus  presented  first  and  the  data  for  sole 
source  contracts  presented  second.  Within  each  category  of 
competed  and  sole  source  contracts;  the  data  is  presented 
for  lower  risk  contracts  and  then  higher  risk  contracts. 
Within  each  risk  category,  both  lower  and  higher  price 
contact  data  is  presented.  The  first  histogram  and  sample 
statistics  are  for  the  entire  sample  of  competed  contracts. 
Subsequent  histograms  and  statistics  result  from  the 
breakdown  of  this  sample  Into  risk  and  initial  price 
categories . 
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Sample  Size  >•  261  Mean  <*  .1444  Variance  -  .0828 


Confidence  Interval  Kstimation.  The  95% 
confidence  interval  for  the  mean  cost  growth  for  competed 
contracts  is  given  below.  Here/  equals  1.96, 

.1444  t  1.96  « 

261 

Lower  Limit  -  .1095  Upper  Limit  -  .1793 


47 


Lower  Risk.  This  data  represents  the  competed, 
lower  risk  category.  The  data  for  the  lower  and  higher 
initial  price  contracts  that  make-up  this  category  follows. 


riguse  8:  Competitive  Category,  Low  Risk 


Sample  Size  "  130  Mean  -  .1278  Variance  ■  .0874 

■CgnCidanga  .Intjgyai  Bgtima.tioni  The  95% 
confidence  interval  for  the  mean  cost  growth  of  lower  risk 
competed  contracts  is  given  below. 


.127  8  i  1.96  * 


Lower  Limit  -  .0770 


.0874 


Upper  Limit  ■  .1785 


Lower  Price.  The  following  are  the  cost 
growth  factors  for  the  lower  initial  price  contracts  within 


figure  9:  Competitive  Cetegery;  Lower  Risk;  Lower  Price 


Sample  Statistics. 

Sample  Size  "  65  Mean  "•  0  Variance  -•  0 

Intiiml-BAUmatign ■  Because 
the  standard  deviation  of  the  sample  is  zero,  there  is  no 
confidence  interval  for  the  true  value  of  the  mean  cost 
growth  for  this  category.  All  65  sample  contracts  in  this 
category  exhibited  zero  cost  growth. 
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Higher  Price.  The  following  are  the  cost 
growth  factors  for  the  higher  initial  price  contracts  within 
the  lower  risk  competed  category. 

.  ffltogmn  Of  Port  aowtHiMki 
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‘  <0.101  <0.110  0.000  o.iiQ  o.ioo  o.oio  i.ooo  i.iio  i.ioo  i.oio  o.ooo 

_ _ _ _ iM.amttt.MSt _ 

figure  10;  Competitive  Category;  Lower  Risk;  Higher  Price 

Sample  Size  -  65  Mean  >  .2929  Variance  -  .1893 

Confidence  Interval  Estimation.  The  95% 
confidence  interval  for  the  mean  coat  growth  for  the  lower 
risk,  higher  price  competed  contracts  is  given  below. 

.2929  i  1.96  • 

6  5 

Lower  Limit  -  .1872  Upper  Limit  .3985 
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Higher  Risk.  Data  are  presented  first  for  the 
competed,  higher  risk  category.  The  stratified  data  for 
lower  and  higher  initial  price  are  than  presented. 


<0.101 

•O.IIO 

0.000 

0.010 

O.iOO 

O.OSD 

1.000 

..110 

1.100 

1.TI0 

o.oeo 

Off 

riguxe  11;  Competitive  Categocy,  Highec  Risk 


Sample  Size  «  131  Mean  -  .1609  Variance  -  .0703 
Confidence  Interval  Estimation.  The  95% 
confidence  interval  for  the  mean  cost  growth  for  the  higher 
risk  competed  contracts  is  given  below. 


.1609  i  1.96  « 
Lower  Limit  -  .1130 


.0783 

131 

Upper  Limit  -  .2087 
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Lower  Price.  The  following  are  the  cost 


growth  factors  for  the  lower  Initial  price  contracts  within 
the  higher  risk  competed  category. 


•o.ni  •o.an  i.soo  t.iio  o.ioo  t.m  i.ioo  i.uo  i.iso  1.711  i.ooa 


777:1, T-r 


rlguse  12:  Compatitivc  Category;  Higher  Risic;  Lower  Price 


Sample  Size  ■■  64  Mean  -  .1191  Variance  -  .0497 


The  95% 


confidence  interval  for  the  mean  cost  growth  for  the  higher 
risk,  lower  price  competed  contracts  is  given  below. 


.1191  t  1.96  * 


.0497 

64 


Lower  Limit  -  .0646 


Upper  Limit  «  .1736 


Higher  Price.  The  following  are  the  cost 


growth  factors  for  the  higher  initial  price  contracts  within 
the  higher  risk  competed  category. 


Sample  Statistics. 

Sample  Size  -  €7  Mean  ■>  .2149  Variance  -•  .1402 


Confidence  Interval  Eatlmation.  The  95% 
confidence  interval  for  the  mean  cost  growth  for  the  higher 
risk,  higher  price  competed  contracts  is  given  below. 

.2149  ±  1.96  * 

Lower  Limit  -  .1253  Upper  Limit  -  .3045 
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_ anflBBttMi _ 

fi9fux«  14:  Sole  Sourot  Contracts 

Sampla  Statiatlea. 

Sample  Size  "  251  Mean  *  .2358  Variance  ■■  .1690 


g,gng44<?ngB-.l!a£iiyfti  Bg^iisaUaiL,.  The  95% 

confidence  interval  for  the  mean  cost  growth  for  sole  source 
contracts  is  given  below.  Again^  Zei^it  equals  1.96. 

.2358  ±  1.96  * 

251 


Lower  Limit  .1850  Upper  Limit  .2865 


54 


Lower  Rialc.  Data  are  presented  first  for  the  sole 


source/  lower  risk  category.  The  stratified  data  for  lower 
and  higher  initial  price  contradts  are  then  presented. 


rigur*  15:  Sole  Source  Category,  Lower  Rislc 

Sample  Size  "  125  Mean  •  .2175  Variance  .1900 
Confidence  Interval  Estimation.  The  95% 
confidence  interval  for  the  mean  cost  growth  for  the  lower 
risk  sole  source  contracts  is  given  below. 

.2175  ±  1.96  * 

125 


Lower  Limit  .1412 


Upper  Limit  -  .2937 


Lower  Price.  The  following  are  the  growth 
factors  for  the  lower  initial  price  contracts  within  the 
lower  risk  sole  sourae~category . 

ISCfrun  of  cost  Growth  iMton 
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Figure  16:  Sole  Source  Cacegory;  Lower  Risk/  Lower  Price 
Sample  ■Statistics... 

Sample  Size  ■■  63  Mean  ■■  .0654  Variance  .04  91 


Confidence  Interval  Estimation.  The  95% 
confidence  Interval  of  the  mean  cost  growth  for  the  lower 
risk,  lower  price  sole  source  contracts  Is  given  below. 


.06S4  1  1.96  « 


.0491 

63 


Lower  Limit  -  .0107 


Upper  Limit  -  .1200 
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Higher  Price.  The  following  are  the  rrowth 
factors  for  the  higher  initial  price  contracts  with  i  the 
lower  risk  sole  source  category. 


Birtoftim  of  Cost  (kinith  Ekton 


t  H  H  m 


Figure  17:  Sole  Source  Category;  Lower  Risic;  Higher  f 

Sample  Statistics. 

Sample  Size  “  62  Mean  .4515  Variance  - 
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! 
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Confidence  Interval  Estimation.  The  95% 
confidence  interval  for  the  mean  cost  growth  of  low  ;  risk, 
higher  price  sole  source  contracts  is  given  below. 


.4515  ±  1.96  * 


.5797 

62 


Lower  Limit  ■  .2619  Upper  Limit  -  .6  0 
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Higher  Rlslc.  Data  are  presented  first  for  the 
sole  source,  higher  risk  category.  The  stratified  data  for 
lower  and  higher  initial  price  contracts  are  then  presented. 


il!l!fllo|nm  d  Coai 
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■r 
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figure  18:  Sort  Source  Category,  Higher  Risk 


Sample  Statistics. 

Sample  Size  ■  126  Mean  -  .2538  Variance  ■  .1488 
Confidence  Interval  Estimation.  The  95% 
confidence  interval  for  the  mean  cost  growth  of  higher  risk 
sole  source  contracts  is  given  below. 


.2538  i  1.96 


.  1499 
126 


Lower  Limit  -  .1866  Upper  Limit  .3210 
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Lower  Price.  The  following  are  the  cost 


growth  factors  for  lower  initial  price  contracts  within  the 
higher  risk  sole  source  category. 


Sample  Statistics. 

Sample  Size  -  63  Mean  “  .1655  Variance  -  .1156 

Confidence  Interval  Estimation.  The  95% 
confidence  interval  for  the  mean  cost  growth  of  the  higher 
risk^  lower  price  sole  source  contracts  is  given  belov;. 

.1655  ±  1.96  * 

63 

Lower  Limit  -  .0816  Upper  Limit  -  .2494 
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Higher  Price.  The  following  are  the  coat 
growth  factors  for  the  higher  Initial  price  contracts  within 
the  higher  risk  sole  source  category. 


Hlsl«|nm  ot  Axi  Groith  Faiiin 


_ flpi  awit  nriM _ 

rlgfuxa  20:  Sole  Source  Category;  Higher  Risk;  Higher  Price 


Sample  Size  ■■  €3  Mean  >■  .3421  Variance  .1687 

Confidence  Interval  Estimation.  The  95% 
confidence  interval  for  the  mean  cost  growth  of  the  higher 
risk,  higher  price  sole  source  contracts  is  given  below. 

.3421  i  1.96  * 

63 

Lower  Limit  -  .2408  Upper  Limit  -  .4434 
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Schedule  Growth  Raw  Data 


Competed  Contraota.  The  following  are  the  schedule 
growth  factors  for  competed  contracts. 


Sample  _Statlstlca. 

Sample  Size  20  Mean  <>•  ,1558  Variance  ■  .0728 
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Sole  Source  Contracts.  The  following  are  the  schedule 


rigure  22;  Schedula  Growth  Factors,  Sole  Source  Contracts 


Sample  Statistics. 

Sample  Size  "•  18  Mean  -  .7083  Variance  >■  1.4244 
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The  following  table  sununarizes  the  cost  data  presented 
earlier.  The  data  are  presented  in  three  phases  that 
correspond  to  the  phases  of  tests  conducted  for  each 
research  question. 


TABLE  3 

PHASE  1:  SUMMARY  DATA 


Competed 

Sole  Source 

N  -  2242 
n  -  261 

mean  growth  -  .1444 

M  -  1260 
n  -  251 

mean  growth  <■  .2358 

TABLE  4 

PHASE  2:  SUMMARY  DATA 


Lower  Risk 

Higher  Risk 

Competed 

N  -  924 
n  -  130 
mean  ■  .1278 

M  -  1318 
n  -  131 
mean  •>  .1609 

Sole  Source 

N  -  548 
n  -  125 
mean  •"  .2175 

N  -  712 
n  -  126 
mean  -  .2538 

TABLE  5 


PHASE  3:  SUMMARY  DATA 


Lower  Risk 

Higher  Risk 

Competed-Lower 
Initial  Price 

n  ■  65 
mean  <>  0 

n  -  64 

mean  -  . 1191 

Competed-Higher 
Initial  Price 

n  ■  65 

mean  .2929 

n  -  67 

mean  ■  .2149 

Sole  Source-Lower 
Initial  Price 

n  -  63 

mean  *•  .0654 

n  ■  63 

mean  »  .1655 

Sole  Source-Higher 
Initial  Price 

n  -  62 

mean  •  .4515 

n  -  63 

mean  «  .3421 
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Thtt  following  table  aununarlzes  the  coat  data  presented 


earlier. 


TABLE  6 


SCHEDULE  GROWTH  SUMMARY  DATA 


Competed 

Sole  Source 

N  -  572 
n  -  20 

mean  growth  •»  .1558 

N  •  198 
n  -  18 

mean  growth  •*  .7083 

Analysis  Results 

The  statistical  analysis  was  performed  for  each 
research  question  and  corresponding  hypothesis  using  the 
large  sample  test  statistic  to  test  for  the  difference  in 
population  means.  The  formula  for  the  large  sample  test 
statistic  is: 


z  ■ 


For  the  schedule  growth  data,  however,  the  large  sample 
test  statistic  is  inappropriate  and  the  Wilcoxon  Ranlc-Sum 
Test  is  used  to  test  for  the  difference  in  population  means. 
The  test  statistic  for  the  Wilcoxon  Ranlc-Sum  Test  is: 

z  «  iLUif 

The  confidence  intervals  of  the  mean  growth  presented 
earlier  provide  invaluable  information  regarding  the 
predictability  of  the  cost  growth  for  a  particular  category. 
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Confldenoft  Intervals  provide  a  ready  referenoe  to  the  range , 
upper  and  lower  bounds,  for  the  mean  cost  growth  with  a 
speoiflad  level  of  signifioance.  For  instance,  referring  to 
the  confidence  interval  for  the  Lower  Rialc/Higher  Price 
Competed  Contracts  on  page  50,  it  oan  be  predicted  that  if 
further  statistical  sampling  was  conducted,  95%  of  the 
samples  would  have  a  mean  cost  growth  between  19%  and  40%. 

The  results  of  the  statistical  analysis  for  each 
researoh  question  and  the  phases  of  tests  that  made-up  each 
research  question  are  now  presented.  Hypotheses  and  sub- 
hypotheses  accompany  the  analysis  for  each  researoh  question 
and  phase  of  test. 

Rifltftgfflh,  Qutatisn  „f.l 

For  both  cost  growth  and  schedule  growth  categories, 
does  the  mean  growth  for  competed  contracts  differ  from  the 
mean  growth  of  sole  source  contract? 

HYfifltitlftaia . 

HOi  Mean  Competed  Growth  -  Mean  Sole  Source  Growth 

Hai  Mean  competed  Growth  *  Mean  Sole  Source  Growth 


For  the  coat  growth  data,  the  analysis  tested  the 
hypothesis  in  three  phases.  The  first  phase  tested  the 
hypothesis  for  the  overall  categories  of  competed  versus 
sole  source  contracts  as  depicted  in  Table  3.  The  second 
phase  tested  the  hypothesis  after  the  sole  source  and 
competed  data  were  segregated  into  lower  and  higher  risk 
categories  as  depicted  in  Table  4.  The  third  phase  tested 
the  hypothesis  after  the  lower  and  higher  risk  data  were 


segregated  Into  lower  and  higher  initial  price  categories  as 
depicted  in  Table  5.  For  schedule  growth  data,  the  analysis 
tested  the  hypothesis  for  the  overall  categories  of  competed 
versus  sole  source  contracts  as  depicted  in  Table  6. 

The  analysis  results  from  testing  Research  Question  #1 
are  presented  first  for  the  cost  growth  and  second  for  the 
schedule  growth.  Recall  that  the  value  of  Za^it  &  two- 
tailed  test  with  a  level  of  significance  of  .05  is  equal  to 
1.96.  Therefore,  the  decision  rule  states  that  if  the  test 
statistic,  Z,  is  either  greater  than  or  less  than 
-Zerit,  the  null  hypothesis  (Ho)  is  rejected  in  favor  of  the 
alternate  hypothesis  (H.) . 

Coat  Growth  Data.  The  results  of  the  cost  growth 
analyses  are  presented  for  the  Phase  1  test  followed  by  the 
2  tests  in  Phase  2  and  finally  the  4  tests  of  Phase  3.  For 
all  Research  Question  #1  tests,  x-bar  and  y-bar  represent 
competed  and  sole  source  mean  cost  growth  respectively. 

Phase  1  Test.  The  raw  data  and  the  analysis  for  the 
overall  categories  of  competed  versus  sole  source  contracts 
are  presented  below, 

X  -  .1444  Si  -  .0828  m  -  261 

y  -  .2358  si  -  .1690  n  -  251 

The  value  of  the  large  sample  teat  statistic,  z  is: 
z  -  -2.903 

The  null  hypothesis  is  rejected  at  significance  level 
.05.  For  the  overall  test  for  cost  growth  between  sole 
source  and  competed  contracts,  the  mean  competed  cost  growth 
is  less  than  the  mean  sole  source  cost  growth. 
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The  statistical  significance  of  this  first  phase  test 
is  that  the  sole  source  cost  growth  is  almost  twice  as  high 
as  competed  cost  growth,  at  23.6%  versus  14.4%.  The  p-value 
required  to  change  the  result  of  this  test  from  rejecting  to 
not  rejecting  the  null  hypothesis  is  equal  to  .004.  In 
other  words,  the  confidence  level  of  the  test  must  equal 
99.6%  in  order  for  the  null  hypothesis  to  not  be  rejected. 

Table  7  summarizes  the  results  of  the  Phase  2  and  3 
statistical  tests  for  the  first  research  question  and  the 
related  hypotheses. 

Phase  2  Tests.  After  Phase  1  testing,  the  raw  data 
were  segregated  into  lower  and  higher  risk  categories.  The 
results  of  these  2  tests  are  now  presented  and  analyzed. 

TABLE  7 

SUMMARY  or  STATISTICAL  TESTS  FOR 


RES 

BARCK  QUESTION 

#1 

Lower  Risk 

Higher  Risk 

Competed  vs . 
Sole  Source 

Do  Mot 
Reject  H, 

Reject  Ho 

Lower  Price 

Reject  Ho 

Do  Not 
Reject  Ho 

Higher  Price 

Do  Not 
Reject  Ho 

Do  Not 
Reject  Ho 

Test  1:  Lower  Risk.  Competed  vs  Sola  Source. 
Hypothesis . 

Hot  Computed  LowBz  Risk  Sols  Source  Lower  Risk 
Hat  Competed  Lower  Risk  *  Sole  Source  Lower  Risk 

The  raw  data  for  these  categories  are: 

X  -  .  1278  si  •  .0874  m  «  130 
y  -  .  217  5  •  1900  ^  "  125 
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The  value  of  the  large  sample  test  statistic,  z  Is: 
z  -  -1.915 

The  null  hypothesis  cannot  be  rejected  at  significance 
level  .05.  The  mean  competed  cost  growth  equals  the  mean 
sole  source  cost  growth  in  the  lower  risk  category.  There 
is  no  statistical  difference  In  the  mean  cost  growth  between 
competed  and  sole  source  contracts  at  the  lower  risk  level. 

Teat_^.; _ va . SOlft 

Hoi  Compoted  Highaz  Risk  •  Solo  Source  Highoz  Risk 

Hai  Compotod  Highoz  Risk  *  Solo  Source  Highoz  Risk 

The  raw  data  for  these  categories  are: 

X  -  .1609  Si  ■  .0782  m  ■  131 

y  -  .2536  Si  >  .1488  b  -  126 

The  value  of  the  large  sample  test  statistic,  z  is: 
z  -  -2.203 

The  null  hypothesis  is  rejected  at  significance  level 
.05.  The  mean  competed  cost  growth  does  not  equal,  and  is 
less  than,  the  mean  sole  source  cost  growth  in  the  higher 
risk  category. 

The  statistical  significance  of  these  second  phase 
tests  are  mixed.  Although  the  lower  risk  hypothesis  could 
not  be  rejected,  the  higher  risk  hypothesis  was  rejected. 
However,  in  both  cases  the  results  were  consistent  In  that 
the  mean  cost  for  competed  contracts  is  considerably  less 
than,  the  mean  cost  growth  of  sole  source  contracts.  The 
first  row  in  Table  8  shows  the  p-values,  or  the  level  of 
significance  for  the  tests,  which  would  conclude  the  null 
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hypothesis,  or  that  the  mean  cost  growth  between  lower  and 
higher  risk  competed  and  sole  source  contracts  are  equal. 

Phase  3  Tests.  After  Phase  2  testing,  the  raw  data 
were  segregated  into  lower  and  higher  initial  price 
categories.  The  results  of  these  4  tests  are  now  presented. 

■TiaA  l\ _ Lower  Risk/Lower  Price  Analysis. 

Hypothesis . 

Hot  Cornpe tod  Lowmx  Sdak/Lowor  Price  > 

Sole  Source  lower  Risk/ Lower  Price 
Hat  Corrtpeted  Lower  Risk/ Lower  Price 
Sole  Source  lower  Risk/ Lower  Price 

The  raw  data  for  these  categories  are: 

3i«0  Sx  •  a  n»"65 

y  -  .0654  Sa  ■  *0491  JJ  ■  63 

The  value  of  the  large  sample  test  statistic,  z  is; 
z  -  -2.344 

The  null  hypothesis  is  rejected  at  significance  level 
.05.  The  mean  competed  cost  growth  does  not  equal  the  mean 
sole  source  cost  growth  for  this  category.  The  test 
indicates  that  the  mean  sole  source  cost  growth  in  the  lower 
risk,  lower  price  category  is  greater  than  the  competed 
contracts  in  the  same  category. 

Tflat  2i _ Lower  Riak/Hicher  Price  Analysis . 

Hoi  Competed  Lower  Risk/ Higher  Price  • 

Sole  Source  Lower  Risk/ Higher  Price 
Hei  Competed  Lower  Risk/ Higher  Price  # 

Sole  Source  Lower  Riak/Higher  Price 
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The  raw  data  for  these  categories  are: 

X  »  .2929  sf  -  .1893  ra  -  65 

y  -  .4515  al  ■  -5797  n  •  62 

The  value  of  the  large  sample  test  statistic,  z,  is: 
z  *  -1,432 

The  null  hypothesis  cannot  be  rejected  at  significance 
level  .05.  The  mean  competed  cost  growth  equals  the  mean 
sole  source  cost  growth  in  the  lower  risk/higher  price 
category. 

Test  3; Hi ghftg . ,Ria,K / ai a 

Hypothesis. 

Ho\  Compmt^d  Higher  Risk/ Lows  Pxic»^ 

Sol 9  Source  Highoz  Riak/Lowoz  Pziae 
Hai  Co/ripeted  Highaz  Riak/Lowoz  Pzicai> 

Sola  Source  Highaz  Riak/Lowaz  Pzica 

The  raw  data  for  these  categories  are: 

X  ■  .1191  a*  ■  .0497  m  ■  64 

y  •  .1655  al  ■  .1156  h  «  63 

The  value  of  the  large  sample  test  statistic,  z,  is: 
z  -  -.9077 

The  null  hypothesis  cannot  be  rejected  at  significance 
level  .05.  The  mean  competed  cost  growth  equals  the  mean 
sole  source  cost  growth  in  this  category. 

Test  4:  Higher  Risk/Hicher  Price  Analysis. 

HY.Bg,thfl8i.a.i. 

Hot  Compatad  High  Riak/Highar  Pzica  ■ 

Sole  Souzce  High  Risk/ Higher  Pziae 
Hat  Competed  High  Risk/ Highez  Pzica  * 

Sole  Souzce  High  Risk/ Higher  Price 
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The  raw  data  for  these  categories  are: 

X  «  .2149  dj  ■  .  1401  in  -  67 

y  «  .3421  Sj  -  .1687  n  ■  63 

The  value  of  the  large  sample  test  statistic,  z  Is: 
z  ■  -1 . 842 

The  null  hypothesis  cannot  be  rejected  at  significance 
level  .05.  The  mean  competed  cost  growth  equals  the  mean 
sole  source  coat  growth  in  this  category. 

The  statistical  significance  of  these  third  phase  tests 
are  mixed.  The  lower  risk/lower  price  category,  test  1,  is 
the  only  teat  that  rejected  the  null  hypothesis.  However, 
in  all  4  cases  the  results  were  consistent  in  that  the  mean 
coat  growth  for  competed  contracts  is  considerably  less  than 
the  mean  cost  growth  of  sole  source  contracts.  The  sc'iond 
and  third  rows  in  Table  8  show  the  p-values,  or  the  level  of 
significance  for  the  tests,  which  would  conclude  the  null 
hypothesis,  that  the  mean  cost  growth  between  lower  and 
higher  initial  price,  lower  and  higher  risk  competed 
contracts  is  equal  to  lower  and  higher  initial  price,  lower 
and  higher  risk  sole  source  contracts. 
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TABLE  8 

P-VALUES  ASSOCIATED  WITH 


RES 

EARCH  QUESTION  #1. 

Lower  Risk 

Higher  Risk 

Competed  vs. 
Sole  Source 

P-Value  -■ 
.055 

P-Value  ■ 
.028 

Lower  Price 

P-Value  ■ 
.019 

P-Value  - 
.363 

Higher  Price 

P-Value  - 
.135 

P-Value  - 
.066 

Schedule  Growth  Data.  Recalling  from  Chapter  III,  for 
the  sole  source  data,  n  -<  18  and  for  competed  data,  n  -  20. 
Using  the  equations  given  in  Chapter  III  for  the  mean  and 
variance  of  W  and  the  above  values  for  m  and  n,  the 
following  values  were  calculated. 

■  351  ai  ■  1170 


The  test  statistic,  W,  is  defined  as  the  sum  of  the 
ranks  of  the  sole  source  data,  from  a  combined  and  ordered 
set  of  competed  and  sole  source  data.  The  value  of  the 
Rank-Sum  W,  was  determined  to  be: 

W  -  394.5 

Using  the  test  statistic  Z  defined  by  Devore  and 
presented  in  Chapter  III,  the  value  of  Z  is: 


2  .  394.5  -  351 


Z  •  1.21 


The  value  of  for  a  two-tailed  test  with  a  .05 

level  of  significance  is  equal  to  1.96.  Therefore,  the 
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decision  rule  states  that  if  Z  is  either  greater  that  Zorit 
or  less  than  the  null  hypothesis  (Hq)  is  rejected  in 

favor  of  the  alternate  hypothesis  (H,)  . 

The  mean  schedule  growth  was  tested  using  the  Wilcoxon 
Ranh-Sum  Test.  The  results  failed  to  reject  the  null 
hypothesis  at  a  significance  level  of  .05.  A  p~value  equal 
to  .204  instead  of  .05  would  result  in  rejection  of  the  null 
hypothesis  and  conclude  that  there  is  a  difference  in 
schedule  growth  between  sole  source  and  competed  contracts. 

Although  the  test  failed  to  reject  the  null  hypothesis 
at  a  significance  level  of  .05,  the  raw  data  indicates  that 
the  schedule  growth  of  sole  source  contracts  is  more  than  4 
times  greater  than  competed  contracts. 

For  the  cost  growth  category  only,  does  the  mean  cost 
growth  for  competed  and  sole  source  lower  risk  contracts 
differ  from  competed  and  sole  source  higher  risk  contracts? 

Ho\  LOW  Risk  Mean  Growth  -  High  Risk  Mean  Growth 
Hai  Low  Risk  Mean  Growth  *  High  Risk  Mean  Growth 

This  analysis  again  used  the  large  sample  test 
statistic  to  test  for  differences  in  population  means 
between  lower  risk  and  higher  risk  contracts,  within 
competed  and  sole  source  categories.  For  cost  growth  data 
the  analysis  tested  the  above  hypothesis  in  2  phases.  The 
first  phase  tested  for  differences  in  the  mean  cost  growth 
between  lower  and  higher  risk  contracts  in  both  competed  and 
sole  source  categories  as  depicted  in  Table  4.  The  second 
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phase  tested  for  differences  in  the  mean  cost  growth  when 
the  Phase  1  data  is  stratified  Into  lower  and  higher  initial 
price  categories  for  both  competed  and  sole  source  contracts 
as  depicted  In  Table  5. 

The  analysis  results  are  presented  first  for  the  2 
Phase  1  tests.  This  is  followed  by  the  results  on  the  4 
Phase  2  tests.  The  decision  rule  stated  earlier  for  a  two-^ 
tailed  test  also  applies  for  this  research  question.  For 
all  Research  Question  #2  tests,  x~bar  and  y-bar  represent 
lower  and  higher  risk  categories  respectively. 

Phase  1  Teats.  The  raw  data  and  analysis  for  lower  and 
higher  risk  contracts  in  both  competed  and  sole  source 
categories,  are  presented  below.  There  are  2  tests. 

Teat  l!  Competed  Category  Overall. 

HygQtheaiai 

HOi  Compeced  Lower  Risk  -  competed  Higrher  Risk 

Hai  Competed  Lower  Risk  *  competed  Hipher  Risk 

The  raw  data  for,  and  the  results  of  the  statistical 

tests  on  the  combined  sample  in  the  competed  category  are: 

X  -  .1278  si  -  .0163  m  -  130 

y  ■  .1609  s|  -  .0782  n  -  131 

The  value  of  the  large  sample  test  statistic,  z  is; 
z  -  -.9287 

The  null  hypothesis  cannot  be  rejected  at  significance 
level  .05.  The  lower  risk  mean  cost  growth  equals  the 
higher  risk  mean  cost  growth  for  competitive  contracts. 

The  statistical  tests  showed  that  the  null  hypothesis 
could  not  be  rejected  at  significance  level  equal  to  .05. 
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There  is  no  statistical  difference  between  competed  lower 
risk  and  competed  higher  risk  cost  growth. 


_ Sole  Source  Category  Overall. 

Hypothesis. 

Hoi  Solo  Source  Lowor  Risk  •  Solo  souxco  Highoi  Risk 
Hai  Solo  Source  Lower  Risk  *  Solo  Source  Highoi  Risk 

The  raw  data  and  the  results  of  the  statistical  tests 
on  the  combined  sample  in  the  sole  source  category  are: 

X  ■  .2175  sj  -  .1900  m  -  125 

y  ■  .2538  al  ■  .1488  n  •  126 

The  value  of  the  large  sample  test  statistic,  z  is; 
z  -  -.6984 

The  null  hypothesis  cannot  be  rejected  at  the  .05 
significance  level.  The  lower  risk  mean  cost  growth  equals 
the  higher  risk  mean  cost  growth  for  sole  source  contracts. 

Statistical  teats  of  Phase  1  showed  the  null  hypotheses 
could  not  be  rejected  at  significance  level  .05,  However, 
in  both  tests  the  cost  growth  on  higher  risk  contracts  is 
greater  than  the  coat  growth  on  lower  risk  contracts. 

Phase  2  Testa.  After  phase  1  testing,  the  raw  data 
were  segregated  into  lower  and  higher  initial  price 
categories.  The  results  of  these  4  tests  are  now  presented. 

The  following  table  summarizes  the  results  of  the 
.statistical  tests  for  Phase  2  of  Research  Question  #2.  The 
testa  determined  if  a  significant  differences  existed 
between  the  mean  cost  growths  of  lower  and  higher  risk 
contracts  when  the  data  was  segregated  into  competed/ sole 
source  and  lower/higher  price  categories. 


TABLE  9 


SUMMARY  OF  STATISTICAL  TESTS  FOR 


RESEARCH  QUESTION  »2 


Lower  Price 

Higher  Price 

Competitive 

Reject  H, 

Do  Not 
Reject  H, 

Sole 

Source 

Do  Not 
Reject  H, 

Do  Not 
Reject  Ho 

T<g,t  It _ Competed  Lower  Price. 

Hy.BgthftAAia.t. 

Hoi  Comps  ted  Lower  Price!  Lower  Hi  sic  • 
CoRtpeted  Lower  Price! Higher  Risk 
Hex  Compete  Lower  price! Lower  Riak  t 
Competed  Lower  price! Higher  Riak 


The  raw  data  for  this  category  are: 

x-0  Sx-0  m»65 

y  ■  .1191  Sa  ■  .0497  n  ■  64 


The  value  of  the  large  sample  test  statistic^  z  is: 
z  -  -4.284 

The  null  hypothesis  is  rejected  at  significance  level 
.05.  The  lower  risk  mean  cost  growth  does  not  equal,  and  is 
less  than,  the  higher  risk  mean  cost  growth  in  the 
competitive,  lower  price  category. 

Iflak-Z.; _ Competed  Higher  Price. 

Hypothesis. 


Hoi  Competed  Higher  Price! Lower  Riak  - 
Competed  Higher  Price! Higher  Riak 
Hei  Competed  Higher  Price! Lower  Riak  * 
Competed  Higher  Price  Higher  Riak 


76 


The  raw  data  for  this  category  are: 

X  ■  .2929  al  -  .1893  m  -  65 

y  ■  .2149  ai  ■  .1401  n  ■  67 

The  value  of  the  large  sample  test  statistic  is: 
z  -  1.102 

The  null  hypothesis  cannot  be  rejected  at  significance 
level  .05.  The  lower  risk  mean  cost  growth  equals  the 
higher  risk  mean  cost  growth  in  this  category. 

3 i _ Sole  Source  Lower  Price. 

Hypothesis. 

Not  Sola  Source  Lower  Price  Lower  Risk  - 
Sola  Source  Lower  Priaa  Highar  Risk 
Hat  Sola  Source  Lower  Prica  Lowar  Risk  * 

Sola  Source  Lower  Price  Highar  Risk 

The  raw  data  for  this  category  are: 

X  >  .0654  Si  "  .0491  m  "  63 

y  -  .1655  s|  -  .1156  n  ■  63 

The  value  of  the  large  sample  test  statistic  is: 
z  -  -1.957 

The  null  hypothesis  is  not  rejected  at  significance 
level  .05.  The  lower  risk  mean  cost  growth  equals  the 
higher  risk  mean  cost  growth  in  the  sole  source. 

4 ; _ Sole  Source  Hiohar  Price. 

flyga-thaaisi 

So:  Sole  Source  Highar  Price/ Lowar  Risk  ^ 

Sole  Source  Higher  Price/ Higher  Risk 
Hat  Sole  Source  Higher  Price/ Lowar  Risk  * 

Sola  Source  Highar  Price/ Higher  Risk 
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The  raw  data  for  this  category  are: 

X  «  .4515  s*  <■  .5797  117  -  62 

y  -  .3421  Sj  -  .1687  13  -  63 

The  value  of  the  large  sample  test  statistic  is: 
z  -  .9975 

The  null  hypothesis  cannot  be  rejected  at  significance 
level  .05.  The  lower  risk  mean  cost  growth  equals  the 
higher  risk  mean  cost  growth  in  the  sole  source,  higher 
price  category. 

The  statistical  significance  of  the  Phase  2  tests  are 
mixed.  The  test  between  lower  and  higher  risk  contracts  in 
the  lower  price  competed  category  resulted  in  the  rejection 
of  the  null  hypothesis.  This  indicates  that,  higher  risk 
contracts  exhibit  greater  cost  growth  than,  lower  risk 
contracts  in  the  competed,  lower  price  category.  For  both 
competed  and  sole  source  contracts,  for  lower  priced 
contracts,  higher  risk  contracts  grow  at  a  higher  rate  than 
lower  risk  contracts.  Conversely,  for  higher  priced 
contracts,  lower  risk  contracts  grow  at  a  higher  rate  than 
higher  risk  contracts,  Table  10  lists  the  p~values,  or  the 
level  of  significance  for  these  4  tests  which  would  conclude 
the  null  hypothesis,  that  the  mean  cost  growth  between  lower 
and  higher  risk  contracts  are  equal  for  categories  of 
competed/sole  source  and  lower  and  higher  initial  price. 
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TABLE  10 


P-VALUES  ASSOCIATED  WITH 


R 

B SEARCH  QUESTION 

#2. 

Lower  Price 

Higher  Price 

Competitive 

P-value  - 
0 

P-Value  - 
.271 

Sole 

P-Value  - 

P-Value  - 

Source 

.05 

.317 

Raaearoh  Question  #3 

For  the  cost  growth  category  only,  does  the  mean  coat 
growth  for  competed  and  sole  source  lower  priced  contracts 
differ  from  competed  and  sole  source  higher  priced 
contracts? 

Hypothesis. 

Hoi  Low  Cost  Mosn  Growth  "  High  Coat  Maan  Growth 

Hat  Low  Coat  Moan  Growth  *  High  Coat  Moan  Growth 

This  analysis  consisted  of  4  tests  for  differences  in 
population  means  between  lower  and  higher  priced  contracts . 
The  analysis  covers  contracts  of  lower  and  higher  risk  for 
both  competed  and  sole  source  categories.  For  all  Research 
Question  #3  tests,  x-bar  and  y-bar  represent  lower  and 
higher  initial  price  cost  growth  respectively.  The  decision 
rile  stated  earlier  also  applies  to  these  tests.  The  value 

of  the  large  sample  test  statistic  Z  was  computed  for  the  4 

phases  of  tests.  Table  11  summarizes  the  results  of  the 
statistical  tests  for  the  third  research  question  and  the 
related  hypotheses. 
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TABLE  11 


SUMMARY  OF  STATISTICAL  TESTS  FOR 
^^^^^__RESEARCHJ^UESTIONJ^__^ 


Lower  Risk 

Higher  Risk 

Competitive 

Reject  He 

Do  Not 
Reject  He 

Sola  Source 

Reject  He 

Reject  Ho 

lt.gA,.Jui _ Competed  ■Category  Lower. Risk  . 

Jig  I, 

/foi  Coaip0fd  Lowmr  Riak/Lowmr  Pzica  - 
Competed  Lower  Risk/ Higher  Price 
Hai  Competed  Lower  Risk/ Lower  Price  t 
Competed  Lower  Riak/Higher  Price 

The  raw  data  for  this  category  are: 

S-O  Si-0  /n«65 

y  •  .2929  3t  -  .1893  h  ■  65 


The  value  of  the  large  sample  test  statistic  is: 
z  -  -5.394 

The  null  hypothesis  is  rejected  at  significance  level 
.05.  The  lower  price  mean  cost  growth  does  not  equal,  and 
is  less  than  the  higher  price  mean  cost  growth  in  the 
competed,  lower  risk  category. 

■Tflat  2; _ Competed  Category  Higher  Risk. 

HYBQthSaigi 

Hoi  Competed  Higher  Risk/ Lower  Price  • 

Competed  Higher  Riak/Higher  Price 
Hai  Competed  Higher  Risk/ Lower  Price  * 

Competed  Higher  Riak/Higher  Price 
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The  raw  data  for  this  category  are: 

X  «  .1191  al  -  .0497  m  ■  64 

y  -  .2149  aji  -  .1401  n  ■  67 

The  value  of  the  large  sample  test  statistic  Is: 
z  •"  “1.788 

The  null  hypothesis  cannot  be  rejected  at  significance 
level  .05.  The  lower  price  mean  cost  growth  equals  the 
higher  price  mean  cost  growth  in  this  category. 

Test  3:  Sole  Source  Category  Lower  Risk. 

ffoi  Sola  Sourca  Lowax  Risk/Lovax  Pxica  ■ 

Sola  Souxca  Lowax  Riak/ Highax  Pxica 
Hai  Sola  Source  Lowax  Riak/Lowar  Pxica  * 

Sola  S^ouJTca  Lowax  Riak  Highax  Pxica 

The  raw  data  for  this  category  are: 

X  «  .0654  a%  •  .0491  m  ■  63 

y  -  .4515  ai  -  .5797  73-62 

The  value  of  the  large  sample  test  statistic  is: 
z  -  -3.836 

The  null  hypothesis  is  rejected.  The  lower  price  mean 
coat  growth  is  not  equal,  and  is  less  than  the  higher  price 
mean  cost  growth  in  this  category. 

T98t  _ Sole  Source  Category  Higher  Risk. 

Hypothesis . 


Hos  Sola  Souxca  Highax  Risk/ Lowax  Pxica  - 
Sola  Souxca  Highax  Risk/ Highax  Pxica 
Hai  Sola  Souxca  Highax  Riak/ Lowax  Pxica  * 
Sola  Souxca  Highax  Riak/Highax  Pxica 


The  raw  data  for  this  category  are: 

X  -  .1655  Si  ■  .1156  m  ■  63 

y  -  .3421  sj  -  .1687  n  -  63 

The  value  of  the  large  sample  test  statistic  is : 
z  ■  -2.628 

The  null  hypothesis  is  rejected  at  significance  level 
.05.  The  lower  price  mean  cost  growth  does  not  equal,  and 
is  less  than  the  higher  price  mean  cost  growth  in  the  sole 
source,  higher  risk  category. 

The  statistical  significance  of  these  4  tests  are 
mixed.  The  competed/higher  risk  hypothesis  is  the  only  test 
that  failed  to  rejected  the  null  hypothesis.  However,  in 
all  4  cases  the  results  were  consistent  with  the  mean  cost 
growth  for  higher  priced  contracts  being  considerably 
greater  than  the  mean  cost  growth  of  lower  priced  contracts . 
Table  12  shows  the  p-values,  or  the  level  of  significance 
for  these  4  tests,  which  would  conclude  the  null  hypothesis, 
or  that  the  mean  cost  growth  between  lower  and  higher 
initial  price  contracts  are  equal  in  the  4  categories 
involved. 


TABLE  12 


P-VALUES  ASSOCIATED  WITH 
RESEARCH  QUESTION  IS. 


Lower  Risk 

Higher  Risk 

Competitive 

P-Value  - 

0 

P-Value  - 
.073 

Sole  Source 

P-Value  - 
.0002 

P-Value  - 
.009 
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The  rasults  conslatently  showed  that  the  competed 
contracts,  in  all  sub-categories,  have  lower  cost  growth 
than  corresponding  sole  source  contracts.  Alsc,  the 
competed  contracts  showed  less  schedule  growth  than  sole 
source  contracts.  Even  though  the  statistical  tests,  in 
most  cases  failed  to  reject  the  null  hypothesis  that  the  two 
means  are  equal,  across  every  category,  the  sole  source  cost 
and  schedule  growth  is  greater  than  the  corresponding  cost 
and  schedule  growth  for  competed  contracts. 

Chapter  V  presents  conclusions  of  both  statistical  and 
managerial  significance  based  on  the  findings  and  analysis 
presented  in  this  chapter.  Recommendations  for  future 
research  that  would  extend  this  research  are  then  discussed. 


% 
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}Ll.  Conclusiona  and  Reconunendationa 


Introduction 

This  chapter  presents  conclusions  based  upon  the 
findings  and  analysis  presented  in  Chapter  ZV.  Discussion 
focuses  on  the  managerial  implications  of  the  analysis  for 
each  research  question.  Finally,  recommendations  for  future 
research  are  suggested. 

Research  Implications 

This  research  was  designed  to  answer  the  3  research 
questions  described  in  Chapter  I.  The  managerial 
significance  of  the  conclusions  are  discussed  for  each 
research  question  and  associated  hypothesis.  This  analysis 
did  not,  nor  did  it  attempt  to  determine  improvements  in 
performance  or  quality,  reduced  technological  risic,  or  the 
strengthening  of  the  defense  industrial  base. 

When  considering  the  managerial  implications  of  the 
statistical  results,  two  factors  must  be  considered.  First, 
the  significance  level  at  which  the  null  hypothesis  is 
rejected  is  strictly  a  management  decision.  If  management 
requires  a  high  level  of  confidence  in  the  test,  a 
possibility  exists  that  the  null  hypothesis  could  be 
accepted  when  in  fact,  it  would  be  rejected  at  a  lower  level 
of  confidence. 

The  second  factor  that  requires  consideration  is  the 
amount  of  variability  in  the  sample  mean.  In  this  research, 
the  high  variability  in  the  sample  mean  cost  growths  caused, 
in  moat  cases,  the  null  hypothesis  to  be  accepted,  when 
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oonsldsrable  differences  existed  In  mean  cost  growths 
between  competed  and  sole  source  contracts. 

Research  Question  #1 

For  both  cost  and  schedule  growth  categories,  does  the 
mean  growth  for  a  competed  contract  differ  from  the  mean 
growth  of  a  sole  source  contract? 

For  this  research  question,  the  conclusions  of  the  cost 
growth  analysis  are  discussed  first  followed  by  the 
conclusions  of  the  schedule  growth  analysis . 

Coat  Growth  Managerial  Implications.  Although  the 
significance  level  selected  prevents  rejection  of  the  null 
hypothesis  In  four  cases,  cost  growths  are  consistently 
higher  on  sole  source  contracts  In  all  categories  of  risk 
and  Initial  price. 

Based  on  the  raw  data  and  analysis  presented  in  Chapter 
IV,  managers  can  expect  sole  source  contracts  to  grow  faster 
than  competed  contracts.  The  analysis  conducted  with  this 
research  reinforces  claims  that  competition  Is  effective  in 
saving  the  government  money  on  Air  Force  procurement's.  The 
prior  studies  discussed  In  Chapter  II  identified  cost 
savings  ranging  from  12%  to  53%  as  a  result  of  competition. 
These  savings  coupled  with  the  savings  Identified  In  this 
study  make  the  overall  dollar  savings  resulting  from 
competition  highly  significant.  It  Is  yet  to  be  determined 
If  these  savings  compensate  for  the  additional  coats 
Incurred  by  the  government  when  using  competition.  However, 
past  performance  does  not  guarantee  future  results.  The 
variability  of  other  effects  on  the  defense  marketplace 
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discussed  in  Chapter  III  may  significantly  impact  the 
effectiveness  of  competition. 

The  data  and  analysis  presented  does  not  support  the 
concept  of  contractor  "buying-in"  discussed  in  Chapter  ll. 

A  contractor  buys-in  to  a  contract  by  bidding  excessively 

low  prices  and  aggressive  schedules  in  hopes  of  recouping  , 

losses  over  time  through  contract  modifications.  This 

practice  is  readily  admitted  by  contractors  however^  this 

analysis  would  support  the  contention  that  buying-in  is  less 

prevalent  than  previously  contemplated.  This  conclusion  is 

reached  because  it  is  counter  intuitive  to  imagine  that 

contractors  would  buy-in  to  sole  source  contracts.  This  may 

be  an  area  for  future  research. 

In  summary/  since  the  growth  rate  of  sole  source 
contracts  is  greater  than  competed  contracts/  the  initial 
savings  of  competition  at  the  time  of  contract  award  does 
not  dissipate  over  time  but  in  fact  the  benefits  of 
competition  are  increased  during  the  execution  of  the 
contract. 

Schedule  Growth  Managerial  Implications.  Although  the 
tests' level  of  significance  does  not  allow  the  null 
hypothesis  to  be  rejected/  analysis  of  the  raw  data 
indicates  that  the  schedule  growth  on  sole  source  awards  is 
over  four  times  greater  than  competed  contracts. 

This  analysis  further  supports  claims  that  competition 
provides  benefits  well  beyond  cost.  When  companies  are  < 

awarded  contracts  under  competitive  conditions,  lower  profit 

I 

margins  force  management  to  ensure  timely  schedule 
performance  because  of  the  schedule  and  cost  relationship. 
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Also,  for  competed  awards,  schedule  and  contractor's  past 
performance  are  usually  part  of  the  source  selection 
criteria.  Because  of  these  factors  contractors  must  more 
closely  scrutinize  and  manage  schedules. 

Finally,  a  possible  explanation  for  the  differences  in 
the  mean  schedule  growth,  the  mean  age  of  sole  source 
contracts  was  more  than  2  years  greater  than  that  for 
competed  contracts.  As  discussed  in  Chapter  III,  the  age  of 
the  contract  may  impact  the  mean  schedule  growth 
calculation.  Contractors  generally  have  more  flexibility 
early  in  a  contract  to  meet  contract  delivery  milestones . 

By  prioritizing  contract  assets,  managers  can  meet  early 
delivery  schedules  at  the  expense  of  later  deliveries. 
Because  schedule  delivery  problems  may  not  be  fully  apparent 
until  later  in  the  contract,  the  competed  contracts  with  a 
lower  average  age  may  not  indicate  the  true  schedule 
performance  of  the  contractor. 

Research  Question  #2 

For  the  cost  growth  category  only,  does  the  mean  cost 
growth  for  competed  and  sole  source  lower  risk  contracts 
differ  from  competed  and  sole  source  higher  risk  contracts? 

Managerial  Implications.  Although  the  level  of 
significance  used  in  the  statistical  tests  prevents 
rejection  of  the  null  hypothesis  in  five  cases,  two 
significant  observations  are  evident  from  the  data. 

1.  For  lower  price  contracts  the  mean  cost  growth  is 
greater  on  higher  risk  contracts  than  on  lower  risk 
contracts,  for  both  sole  source  and  competitive  categories. 
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2.  For  higher  price  contracts  the  mean  cost  growth  is 
less  on  higher  risk  efforts  than  on  lower  risk  contracts/ 
for  both  sole  source  and  competitive  categories. 

For  lower  price  contracts  the  raw  cost  growth  data 
intuitively  makes  sense.  It  should  be  expected  that  the 
cost  growth  on  a  contract  with  higher  risk  would  grow  faster 
than  a  lower  risk  contract  regardless  of  the  procurement 
strategy.  For  this  research/  higher  risk  contracts  are 
defined  as  any  coat  type  contract.  Because  a  cost  contract 
is  used/  the  contractor  does  not  have  the  same  financial 
pressures  as  those  found  on  a  fixed  price  contract. 

For  higher  price  contracts/  the  cost  growth  raw  data  is 
difficult  to  analyze,  k  possible  explanation  for  this  set 
of  data  may  be  associated  with  the  operational  definitions 
for  higher  and  lower  risk  contracts  used  in  this  thesis. 
Since  the  research  defined  higher  risk  efforts  as  some  type 
of  fixed  priced  contracts/  the  successful  offerors  may  have 
accounted  for  the  risk  in  the  effort  with  their  initial 
proposals.  Conversely/  on  lower  risk  efforts/  a  relatively 
higher  degree  of  competition  may  exist  because  more  sources 
are  qualified  to  complete  the  effort  being  contracted  tor, 
thus  the  initial  awards  may  have  tighter  margins . 

#,3 

For  the  cost  growth  category  only/  does  the  mean  cost 
growth  for  competed  and  sole  source  lower  priced  contracts 
differ  from  competed  and  sole  source  higher  priced 
contracts? 
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Managerial  Implications.  Again,  the  factors  of 
significance  level  and  variability  have  important  management 
implications.  Analysis  of  the  raw  data  shows  that  the  mean 
coat  growth  is  greater  on  higher  price  contracts  than  on 
lower  price  contracts  for  all  categories.  Three  possible 
»  explanations  of  the  results  are  offered: 

1.  Higher  priced  contracts  are  more  complex  than  lower 
priced  contracts  thus  are  subject  to  higher  cost  growth. 

Just  the  fact  that  an  effort  is  high  priced  may  eliminate 
qualified  sources  from  the  competition.  The  possibility  of 
a  loss  on  a  higher  priced  effort  may  destroy  the  company, 
thus  management  elects  to  pass  up  the  opportunity  to  bid. 

2.  Lower  priced  contracts  have  a  higher  level  of  direct 
and  indirect  competition  than  higher  priced  contracts.  In 
other  words,  on  lower  priced  efforts,  offerors  not  only 
compete  against  each  other  but  the  program  itself  is 
competing  against  other  items  in  the  budget. 

3.  Lower  priced  contracts  do  not  have  the  same 
flexibility  to  expand  in  scope  as  the  higher  priced 
contracts.  Lower  priced  contracts  may  have  a  limited  scope 
and  do  not  have  opportunities  to  expand.  Also,  large 
contracts  develop  their  own  constituency  over  time  because 
of  the  civilian  jobs  tied  to  the  project.  This  gives  the 
project  a  life  beyond  its  military  utility  or  support. 

Recommendations  for  Future  Research 

This  thesis  investigated  the  cost  and  schedule  growth 
«  of  sole  source  and  competitive  contracts  relative  to  risk 

and  initial  price.  The  risk  and  coat  stratification  is  only 
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one  of  many  possible  ways  to  segregate  the  growth  rates  of 
competitive  and  sole  source  awards.  Variables  that  cruld  be 
studied  to  increase  the  body  of  knowledge  on  the  "benefits 
of  competition"  are: 

1.  Comparisons  of  cost  and  schedule  growth  among 
different  product  centers  and/or  weapon  systems.  This  may 
give  the  government  insight  to  the  competitiveness  of 
different  segments  of  the  defense  Industry. 

2.  Comparisons  of  cost  and  schedule  growth  relative  to 
changes  in  the  defense  budget.  This  analysis  may  give  the 
government  insight  to  how  overall  defense  expansion  or  cuts 
ripple  through  the  defense  industry.  This  may  measure  the 
effects  of  program  stretch  out,  industry  capacity  or  the 
competitive  effects  of  competing  programs. 

3.  Comparisons  of  cost  and  schedule  growth  before  and 
after  the  implementation  of  CICA.  This  study  may  help  to 
clarify  the  true  effects  of  CICA. 

4.  Comparisons  of  cost  and  schedule  growth  over  time. 
Analysis  of  how  cost  and  schedule  growths  have  changed  in 
increments  of  five  years,  form  the  early  1970' s  to  the 
present  would  possibly  provide  a  predictive  ability  for 
future  cost  and  schedule  growths . 

5 .  Comparisons  of  cost  and  schedule  growth  during 
specific  acquisition  phases.  This  would  give  the  Air  Force 
insight  into  the  effects  of  acquisition  strategies  on  cost 
and  schedule  growth  and  how  decision  early  in  a  programs 
life  can  effect  the  level  of  competition  during  the  later 
phases  of  development . 
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6.  Comparisons  of  the  qualitative  aspects  between 
competed  and  sole  source  contracts.  Although  research  on 
competition  has  focused  mainly  on  cost  benefits^  it  seems 

intuitively  obvious  that  benefits  can  be  gained  on  weapon 

r 

system  performance  and  quality  with  the  introduction  of 
«  competition. 

7.  A  regression  model  could  be  developed  to  predict 
cost  and  schedule  growth  based  on  a  number  of  contract 
parameters.  For  example  contract  descriptive  data  such  as> 
acquisition  strategy /phase,  number  of  competitors,  type  of 
contract,  contract  age,  initial  price,  and  type  of  weapon 
System/Product  Center  could  be  used  as  parameters  in  a 
regression  model.  This  can  be  used  by  Air  Force  managers 
for  budget  planning  or  as  a  tool  to  develop  acquisition 
strategy . 

The  future  research  possibilities  on  this  subject  are 
numerous.  The  accessibility  to  valid  contract  information 
from  the  AMIS  data  bases  expands  the  research  possibilities 
and  enables  relatively  easy  data  retrieval  and  analysis. 
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